(gdb) break *0x972

Ten years ago, I started university and learned how to program and use Linux. A couple of years after, I kicked Windows (XP) out of my computers, and never get back to it since.

I'd like to come back on one aspect that makes me love Linux-based environments: they are easily to program (but by easy, I only include myself, I know it's not true for everyone!).

Computers are good at repeating things and following rules. That's the only thing they actually do, so I want to be able to exploit it, and I do it quite often, not just as part of my development work, but also to improve my daily use of my computer.

Let me give some examples.

Composability

Composability is ability to connect different tools together. That's actually part of the "Unix Philosophy", combining "small, sharp tools" to accomplish larger tasks. What did I last use ?

• git log | grep timovn | head -15 List the last 15 git commits of user timovn
• ps aux | grep apache List the processes running Apache webserver
• find /var/lib/mysql/$dbname -type f -exec stat --format '%Y +%n' {} \; | sort -nr | cut -d+ -f2 | head -1. This one is more complex ...
  • find $DIR -type f -exec stat ... {} find all the files in $DIR and execute command stat
  • stat --format '%Y +%n' $FILE print the file timestamp (%Y), the sign + and the file name
  • sort -nr sort numbers (ie, 2 before 10), in reverse order
  • cut -d+ -f2 Split on the "+" sign and take the second member
  • head -1 take the first line of that and that gives use the most recent file of directory /var/lib/mysql/$dbname ! (that's copied from below)

Linux/Unix tools are great for that, although when it's become too complicated, it may be worth starting writing a bash script, or even a Python script. Just to name of few of them:

• ls list the files of a directory
• grep grep a word in a text ... wait, grep is not an english word ... from their man page "grep searches the named input FILEs for lines containing a match to the given PATTERN"
• ssh connect to a remote computer, and do everything you can do on your console in another computer. Great to help your significant other, but requires a bit of configuration to help your parents on the other side of earth.
• wget download a webpage
• head, tail, sort, cut: select top or bottom lines, sort the lines, select one part of a line

Software design with such usages in mind also tend to export some of their functionalities to the command line. For instance, Dia or LibreOffice:

$ dia --help
Usage:
  dia [OPTION…] [FILE...]

Application Options:
  -e, --export=OUTPUT                   Export loaded file and exit
  -t, --filter=TYPE                     Select the filter/format out of: png, jpg, svg, tex, ...
  -s, --size=WxH                        Export graphics size

$ libreoffice --help
LibreOffice 4.3.6.2 430m0(Build:2)
--convert-to output_file_extension[:output_filter_name[:output_filter_options]] [--outdir output_dir] files
      Batch convert files.
      If --outdir is not specified then current working dir is used as output_dir.
      Eg. --convert-to pdf *.doc
          --convert-to pdf:writer_pdf_Export --outdir /home/user *.doc
          --convert-to "html:XHTML Writer File:UTF8" *.doc
          --convert-to "txt:Text (encoded):UTF8" *.doc
--print-to-file [-printer-name printer_name] [--outdir output_dir] files
      Batch print files to file.
      If --outdir is not specified then current working dir is used as output_dir.
      Eg. --print-to-file *.doc
          --print-to-file --printer-name nasty_lowres_printer --outdir /home/user *.doc

Repetitive things

• Last week, I had to 1/ modify a file 2/ save it 3/ go to a console window 4/ hit the up key (ie, go to previous command) and 5/ hit enter to generate an image.

  Points 1/ and 2/ are not that repetitive, but 3-5 are. So let's see if we can automatize this a bit.

  inotifywait is the tool for that!

  $ man inotifywait NAME inotifywait - wait for changes to files using inotify DESCRIPTION inotifywait efficiently waits for changes to files using Linux's inotify(7) interface. It is suitable for waiting for changes to files from shell scripts. It can either exit once an event occurs, or continually execute and output events as they occur. OUTPUT inotifywait will output diagnostic information on standard error and event information on standard output. The event output can be configured, but by default it consists of lines of the following form: ...

  So I wrote this little script:

  $ cat ~/bin/autorun
  #!/bin/bash    
  WATCH=/tmp/autorun
  
  touch $WATCH
  while [[ -f $WATCH ]]
  do
      echo Run $*
      $*
      echo Waiting ...
      inotifywait $WATCH -qq
  done
  echo Done
  

  and I combined it with i3 bindings:

  bindsym F9 exec touch /tmp/autorun
  bindsym Shift+F9 exec rm -f /tmp/autorun
  

  and finally, in my console, I just run $ autorun $MY_COMMAND, and hit F9 to trigger my command execution (that was more convenient/easier than a compile-on-save).

  That's nothing complicated, advanced, nor really meant to reuse after I forget about it, but it surely saved my some time and concentration (it's not the few seconds I took to do it manually that are the problem, but rather that I had to get out of my problem during that time and possibility lost my thread of through).

• I wanted daily backups of my MySQL databases, into separate files, keeping the last 7 backups, but only if the database was modified since the last backup.

  # password-less connection (with sudo)
  # ask MySQL to list its databases
  # excluse system databases
  databases=$(mysql --defaults-file=/etc/mysql/debian.cnf -e "SHOW DATABASES;" | grep -Ev "(Database|information_schema|performance_schema)")
  
  for db in $databases; do
    dbname=$(echo $db | sed 's/\./@002e/g') # points in databases complicate everything ...
    # in /var/lib/mysql/$dbname dir, find the most recent table
    most_recent_db_file=$(find /var/lib/mysql/$dbname -type f -exec stat --format '%Y +%n' {} \; | sort -nr | cut -d+ -f2 | head -1)
  
    back=db_$db.sql
    # if the backup file doesn't exist or the database is newer than the backup
    if [[ ! -f $back.0 || $most_recent_db_file -nt $back.0 ]]; then
      # dump database $db into backup file $back
      mysqldump --force --opt --databases $db > $back
      # debian tool that saves/deletes/compresses backup files $back[.gz]{.0-7}
      savelog -c 7 $back > /dev/null
    fi
  

  done

Programmable tools

When software tools evolve again and again, they accumulate a lot of functionalities that are provided to their users. And frequently, these functionalities can be combined to create "higher level" functionalities. Either you click on the first feature, configure it and run it, and the same for the second, and the third, and you start over when you want to replay the high-level feature. Or your tool can be programmed (or scripted, if it differs).

• Gimp can be scripted with Python or its Scheme dialect (functional programming)
• GDB can be scripted with Python, Guile or its command-line commands. The Python interface is very powerful (Guile is quite new, I never tried it). To give an idea of what I mean, I build all of my PhD thesis tool, mcGDB, using that interface. This means that, out of GDB's default functionalities, I built a brand new tool, original in the research sense. No one though of that before (hopefully), but thanks to GDB's generic interface, I could develop what ever I had in mind.
• I'm not sure it's fair to put console shells like bash or zsh in this list, but they are obviously great at being scripted! Bash scripting evolved a lot since the traditional sh capabilities, and shell scripts are massively used in Unix environments.

• Emacs can be configured and extended with through its Lisp engine. What do I do with that?

  1/ easy one, copied from the Internet (Ctrl-C Ctrl-X quits Emacs)

  ;; ask confirmation before exiting
  ;; C-x C-c are soo close, y'know what I mean ...
  (defun confirm-exit-emacs ()
          "ask for confirmation before exiting emacs"
          (interactive)
          (if (yes-or-no-p "Are you sure you want to exit? ")
                  (save-buffers-kill-emacs)))
  
  (global-unset-key "\C-x\C-c")
  (global-set-key "\C-x\C-c" 'confirm-exit-emacs)
  

  Again nothing complicated, I'm not happy with the default behavior, quiting is too easy if everything is saved (otherwise it asks for confirmation), I can change it myself. Here I just add a confirmation, but I could do what ever I want: delete temporary files, log the time I stopped working, call my mum, ...

  2/ In my configuration, Shift+Tab tries to complete the current word with words found in other buffers (ie open files). While writing my PhD thesis, I used that:

  (defun load-english-completion ()
     "Load english dictionary in background for dynamic completion"
    (interactive)
    (find-file "~/Documents/redactions/english")
    (make-buffer-uninteresting)
    (switch-to-prev-buffer))
  

  ~/Documents/redactions/english is a dump of all the words known by aspell spell checker. (make-buffer-uninteresting) adds a space in front of the buffer name, so that it doesn't appear in the buffer lists.

  3/ For compiling Latex documents, the default configuration in C-x C-s (save buffer) C-c C-c Enter (run "LaTeX" command). That's too long for me, so:

  ;; compile LaTeX document without any confirmation
  (defun LaTeX-do-build ()
    (interactive)
      ;; save current Latex documents
      (let ((TeX-save-query nil))  (TeX-save-document (TeX-master-file))) 
      ;; run LaTeX command on the master file
      (setq build-proc (TeX-command "LaTeX" 'TeX-master-file -1)) 
      (set-process-sentinel  build-proc  'build-sentinel))
  
  (global-set-key (quote [f5]) 'LaTeX-do-build)