#+TITLE: My Solution To Dotfile Management #+AUTHOR: Mitch Tishmack #+STARTUP: hidestars #+STARTUP: odd #+BABEL: :cache yes #+PROPERTY: header-args :tangle yes :cache yes :comments no :padline no In the third age of my dotfile setup (2015), I used xstow and update-dotdee to construct my dotfiles from tiny pieces of files as a sort of mini templating engine. I have since evolved to this setup. Originally the intention was to make it easy for me to compile dotfiles based on certain parameters. Aka, my dotfiles are the same for x, y, and z, whatever those are, but for certain cases z should have one section be different or added etc... as the case may be. Aka it was just a glorified program or templating system. Since I already use org-mode to build my emacs configuration, why not just do the same for the dotfiles. So now if you want to look at my dotfiles, they're all here. * Use/Abuse Use of this setup is somewhat simple or straightforward. There really is one step in general: #+BEGIN_SRC sh :tangle no make #+END_SRC ** Emacs functions to add to init.el For all this to work in emacs so you can edit source blocks you'll need add the following two functions otherwise you'll get an error every time you try to edit the source block. #+BEGIN_SRC emacs-lisp :tangle no (defun tangle/yn (p) (if (bound-and-true-p p) "yes" "no")) (defun tangle/file (file p) (if (eval p) (concat "tmp/" file) "no")) #+END_SRC This is also all the "magique" that this whole stupid repo entails. But how would you control how things get exported? Well that is done via files with simple *(setq predicate-p t)* lines inside them. These are all the *options/name.el* files comprise of. But lets say you want to export things to another directory than $HOME. No worries, just specify DEST: #+BEGIN_SRC sh :tangle no make DEST=/some/other/directory #+END_SRC But wait, what if you want to export things for an os you're not using? Easy, just create a couple options files like so: #+BEGIN_SRC sh :tangle no name=option echo "(setq ${name}-p t)" > options/${name}.el echo "(setq ${name}-p nil)" > options/no-${name}.el #+END_SRC Or, more easily/lazily using the option makefile target: #+BEGIN_SRC sh :tangle no make option NAME=predicate-name-without-p #+END_SRC And then provide the *name* (without the .el extension) to make via *OPTS*: #+BEGIN_SRC sh :tangle no make DEST=/some/other/directory OPTS=macos #+END_SRC This lets you customize things to a ludicrous degree at runtime like so: #+BEGIN_SRC sh :tangle no make OPTS=no-nix:linux:tmux:git:no-zsh #+END_SRC Etc... I wouldn't do this exactly, while you could, thats just too much to need to remember to type. The simplest way to do things is to just create an *options/$(uname -n).el* file with all the options you need defined. Note that in the or *BEGIN_SRC* lines you can use elisp to control when/how something should or should not get tangled. Example use of tangle/file, note the final option is just elisp and we just use *bound-and-true-p* to detect if we have a predicate or not. #+BEGIN_SRC emacs-lisp :tangle no (tangle/file "some/file/name" (bound-and-true-p macos-p)) (tangle/file "some/file/name" (bound-and-true-p foo-p)) (tangle/file "some/file/name" (and (bound-and-true-p first-p) (bound-and-true-p second-p))) #+END_SRC By default the makefile will add an option with the name of the system its running on's *uname -n*. To override this behavior use the *USEROPTS* variable instead, or invoke things with a null *HOST* like so: #+BEGIN_SRC sh :tangle no make HOST= #+END_SRC Want to copy a specific generation to somewhere else? Or maybe you tested installing to some *DEST*, and want to copy that to *$HOME*? No worries: #+BEGIN_SRC sh :tangle no make copy GEN=N #+END_SRC Where N is the generation you want to copy. Note, to ensure this doesn't by default overwrite hand edited files, a diff is run across the files that would be copied to and what the tangled version contains. Example, I've commented out some .gitignore lines in *~/.gitignore* manually and then tried to tangle a new generation over the top. #+BEGIN_SRC sh :tangle no ./ddiff /Users/me/src/github.com/mitchty/dotfiles/generation/291 /Users/me --- .gitignore 2017-03-04 12:49:17.000000000 -0600 +++ /Users/me/.gitignore 2017-03-04 14:00:24.000000000 -0600 @@ -8,6 +8,6 @@ *.pyc *.rbc *.elc -*.swp -*.[oa] -*.hi +#*.swp +#*.[oa] +#*.hi differences between /Users/me/src/github.com/mitchty/dotfiles/generation/291 and /Users/me #+END_SRC This is an attempt to ensure that we don't accidentally overwrite files that may have manual customizations in them. To forcibly overwrite the files just run *make copy GEN=N* with *N* as the number to force overwriting the destination files. ** How does it work? It is really rather simple, the [[file:Makefile][Makefile]] isn't that complex. Look at that for details. This isn't intended to cover everything. This could be considered a template for how you could setup your files in a similar way. Look at this org mode file for details. ** Removal of files between generations Removal of files is ultimately *YOUR* job. However... to make it possible to remove things if desired a bit easier you can run this makefile target with the *OLD* and *NEW* variables set: #+BEGIN_SRC sh :tangle no make removed OLD=generation NEW=generation #+END_SRC This will list all the files/directories recursively not in *OLD* compared to *NEW*. Example: #+BEGIN_SRC sh :tangle no $ make removed OLD=40 NEW=470 .Xdefaults .Xresources .ghc.hs escp essh flushdns tomod towip #+END_SRC ** Explanation of what is happening This allows me to tangle files that would be useful for linux/bsd/etc... without affecting the existing files. The general idea is this (look at Makefile for details): - increment generation count from last generation - tangle files to tmp/$filename based on current settings - copy tmp to generation/N - iff generation/(N-1) exists, diff each file there to what exists at DEST - If diffing fails, the destination files have been updated, STOP, might lose hand edited changes. Note, if destination does not exist, this is ignored. Also setting FORCE will ignore this check. - If diffing does not fail, hardlink generation/N/$files to DEST/$files - Update last with current generation. Note, the destination can be anywhere, not just $HOME. This allows one to compile/tangle files that can then be trivially rsynced to remote machines, or to tar/xz the files as needed. The key here is emacs is only required to generate config files, not necessarily to use them. ** How would I use this? Should be easy enough to either clone this repo or copy things to a new repo and hack in what you need. Your call. But, say you have a heading, take .profile as an example, under an org mode heading you would just add to your *BEGIN_SRC* definition like so: #+BEGIN_SRC text :tangle no #+BEGIN_SRC sh :tangle .profile stuff #+END_SRC #+END_SRC Then any source blocks for that heading will go to *tmp/.profile*. Its important that you put everything into *tmp*! This is used to generate things before a generation is built. It gives the Makefile a chance to know if the tangling worked or not. But lets say you don't need to have lots of sub headings, or even control a file in multiple subparts that have predicates to control things. Pretty simple, just add a source block like normal: #+BEGIN_SRC text :tangle no #+BEGIN_SRC conf :tangle tmp/.example.conf put contents here! #+END_SRC #+END_SRC * External Tanglers Putting everything in readme.org was getting annoying. So started to split things apart. Org links to all the | name | file | |----------+----------------| | emacs | [[file:emacs.org][emacs.org]] | | tmux | [[file:tmux.org][tmux.org]] | | git | [[file:git.org][git.org]] | | x | [[file:x.org][x.org]] | | nix | [[file:nix.org][nix.org]] | | zsh | [[file:zsh.org][zsh.org]] | | vim | [[file:vim.org][vim.org]] | | misc | [[file:misc.org][misc.org]] | | .profile | [[file:dotprofile.org][dotprofile.org]] | | ~/bin | [[file:bin.org][bin.org]] | Language specific | name | file | |---------+-------------| | haskell | [[file:haskell.org][haskell.org]] | | perl | [[file:perl.org][perl.org]] | * TODO Enhancements [2/5] :someday: - [X] Figure out some way to make code blocks editable with :tangle, it sucks not being able to edit blocks as they are. - [X] Need to have some way to autocleanup old generations. Rm works for now so meh. - [ ] Need to add the ability to detect that make is generating a pointless new generation. Aka generation N and generation N-1 are the same, just leave N and don't increment. - [ ] Maybe checksum file contents somehow and use that? - [ ] More? For now its functional. * Reference for babel stuff Found this STUPID useful for constructing the tangle stuff. [[https://raw.githubusercontent.com/eschulte/babel-dev/master/scraps.org][babel scraps link]]