Steinar Bangs blogg<p><strong>version control of /etc with git and git-store-meta</strong></p><p> This blog post describes how to use <a href="https://git-scm.com" rel="nofollow noopener" target="_blank">git</a> together with <a href="https://github.com/danny0838/git-store-meta" rel="nofollow noopener" target="_blank">git-store-meta</a> to version control config files in unix/linux <code>/etc</code> directories </p> <p><strong>Intro and backstory</strong></p> <p> I have used <a href="https://en.wikipedia.org/wiki/Version_control" rel="nofollow noopener" target="_blank">version control</a> on the config files in my unix/linux home directories, since the early 90-ies, and used version control on config files in the <code>/etc</code> directory tree of my linux boxes, since I first started setting up linux boxes in the mid-late ninties. </p><p> Initially I used <a href="https://en.wikipedia.org/wiki/Revision_Control_System" rel="nofollow noopener" target="_blank">RCS</a> both for home directory version control. </p><p> But while <a href="https://en.wikipedia.org/wiki/Hidden_file_and_hidden_directory" rel="nofollow noopener" target="_blank">dotfile</a> version control in my home directories transitioned from <a href="https://en.wikipedia.org/wiki/Revision_Control_System" rel="nofollow noopener" target="_blank">RCS</a> controlled files in home directories on various computers, to a shared <a href="https://en.wikipedia.org/wiki/Concurrent_Versions_System" rel="nofollow noopener" target="_blank">CVS</a> repository used in all home directories, which was later transformed from CVS to <a href="https://en.wikipedia.org/wiki/Apache_Subversion" rel="nofollow noopener" target="_blank">SVN</a>, until it was finally converted from SVN to <a href="https://git-scm.com" rel="nofollow noopener" target="_blank">git</a> in May 2011, I continued to use RCS versioning for files in <code>/etc</code> directories on various computers. </p><p> The reason for staying with <a href="https://en.wikipedia.org/wiki/Revision_Control_System#Behavior" rel="nofollow noopener" target="_blank">RCS</a> version control for files in <code>/etc</code> directories was git’s lack of metadata storage, in particular user and group membership and user and group access to files, as well as limiting public access to files. </p><p> To some extent this was handled by RCS (e.g. by giving the version control file the same owner and group membership as the target file), but it always required a bit of manual fiddeling when things broke. </p><p> This blog post explains how to version control <code>/etc</code> directories by combining <a href="https://git-scm.com" rel="nofollow noopener" target="_blank">git</a> with <a href="https://github.com/danny0838/git-store-meta" rel="nofollow noopener" target="_blank">git-store-meta</a> to preserve user and group membership of the versioned files as well as access privileges to the files and even preserve the time of the last change to the files. </p> <p><strong>How to set up version control of /etc directories</strong></p> <p> The first thing to do is to install the prerequisites. On a debian (or ubuntu) system that could be done by doing the following commands as root: </p><p><em></em></p><pre>apt updateapt install git perl</pre><p> Clone the git-store-meta git repository (this can be done by any user. Doesn’t need root privileges): </p><p><em></em></p><pre>cd /tmpgit clone https://github.com/danny0838/git-store-meta.git</pre><p> Then create an empty git repository in the /etc directory. Do the following commands as root </p><p><em></em></p><pre>cd /etcgit --init</pre><p> Rename the default branch (typically <code>master</code> or <code>main</code>) to a branch name reflecting the hostname of the machine containing the <code>/etc</code> directory being versioned, e.g. if the host is named <code>doohan</code>, the branch should be named something like <code>etc-doohan</code>: </p><p><em></em></p><pre>git branch -M master etc-doohan</pre><p> Add an <code>/etc/.gitignore</code> file ignoring everything (which means that version control of new files has to be done by forcibly adding the files with “<code>git add -f</code>“): </p><p><em></em></p><pre>cd /etcecho "*" >.gitignoregit add -f .gitignoregit commit -m "Ignore all unknown files found in /etc and subdirectories"</pre><p> Copy git-store-meta into the git repository of <code>/etc/.git</code> and create an initial metadata file with the necessary fields for controlling and versioning ownership and access of files in <code>/etc</code> </p><p><em></em></p><pre>cd /etccp /tmp/git-store-meta/git-store-meta.pl .git/hooks/.git/hooks/git-store-meta.pl --store -f user,group,mode,mtime,atimegit add -f .git_store_metagit commit -m "Add git-store-meta metadata file"</pre><p> Set up the git-store-metadata hooks which will update <code>.git_store_meta</code> on commits and add <code>.git_store_meta</code> to the commit and will set metadata values on files when updating a branch: </p><p><em></em></p><pre>cd /etc.git/hooks/git-store-meta.pl --install</pre><p> Add a remote (make sure it is not a public repository) and push the branch: </p><p><em></em></p><pre>git remote add origin git@git.example.com:~exampleuser/etc-configgit push -u origin HEAD</pre> <p><strong>Using git version control of /etc for a new file</strong></p> <p> In the future, when doing changes to a new config file, do the following: </p><ol><li><p> Add the current version of the file to git (the “-f” flag is necessary because of the .gitignore file excluding all files by default) </p><p><em></em></p><pre>cd /etcgit add -f dnsmasq.confgit commit -m "Add version of dnsmasq.con distributed by debian 13 trixie"</pre></li><li><p> Make local modifications to the configuration files and commit the changes and push the branch to the remote for safekeeping </p><p><em></em></p><pre>cd /etcgit add dnsmasq.confgit commit -m "Adapt dnsmasq to home network"git push</pre><p> (note that the “-f” flag isn’t necessary on “git add” here) </p></li></ol> <p><strong>Preserving history from RCS files</strong></p> <p> When adding git versioning to <code>/etc</code> I preserved the history from existing the RCS version control files on the various computers, by </p><ol><li><p> Creating a CVS repository based on RCS files from <code>/etc</code> </p><p><em></em></p><pre>mkdir /tmp/etc-cvscd /etcfind . -name RCS | xargs tar cf - | (cd /tmp/etc-cvs; tar xf -)</pre></li><li><p> installed <a href="https://gitlab.com/esr/cvs-fast-export" rel="nofollow noopener" target="_blank">cvs-fast-export</a> on the computer where the conversion was run (must be done by root or sudo) </p><p><em></em></p><pre>apt install cvs-fast-export</pre></li><li><p> exported the cvs repo to a fast-import file and then imported that file into an empty git repository </p><p><em></em></p><pre>cd /tmp/etc-cvs/find -type f | cvs-fast-export >/tmp/etc.dumpmkdir /tmp/etc-configgit initgit fast-import </tmp/etc.dumpgit branch -m master etc-doohangit config user.name "root doohan"</pre></li><li><p> moved the new .git directory to <code>/etc</code> </p><p><em></em></p><pre>cd /etcmv /tmp/etc-config/.git .</pre></li><li>then added git-store-meta metadata versioning and pushed the branch to a remote, as outlined in the previous section</li></ol> <p><strong>Closing words</strong></p> <p> I also added git-store-meta metadata versionining to git versioning of my various home directories, which removed the need to manually do “<code>chmod go-rwx</code>” on files that should be unaccessible to others after switching branches or merging branches. </p><p> When I first looked into using git for <code>/etc</code> version control, back in 2018, I had planned to use a tool named <a href="https://github.com/przemoc/metastore" rel="nofollow noopener" target="_blank">metastore</a> to store the metadata of <code>/etc</code> directory. </p><p> However, metastore proved to be unsatisfactory for adding metadata support to git: </p><ol><li>metastore added metadata for all files in the <code>/etc</code> directories and subdirectories, not just the files version controlled by git, which caused git updates and commits to become very slow</li><li>metastore metadata files were binary, which isn’t optimal for git versioning of the files or visual inspection of commits</li><li>the last commit was from February 1 2023, which isn’t too long ago, but the last release of metastore was version 1.1.2 on January 6 2018, which <i>is</i> a long time ago</li></ol><p> In contrast <a href="https://github.com/danny0838/git-store-meta" rel="nofollow noopener" target="_blank">git-store-meta</a> has can be used from its main HEAD, rather than a specific release, was easy to install, only tracked metadata for files in git in a human readable <a href="https://en.wikipedia.org/wiki/Comma-separated_values" rel="nofollow noopener" target="_blank">CSV</a> file, and worked well (i.e. does what it is supposed to do without breaking anything in the process). </p> <p><a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://steinar.bang.priv.no/tag/config/" target="_blank">#config</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://steinar.bang.priv.no/tag/configfiles/" target="_blank">#configfiles</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://steinar.bang.priv.no/tag/cvs/" target="_blank">#cvs</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://steinar.bang.priv.no/tag/etcdir/" target="_blank">#etcdir</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://steinar.bang.priv.no/tag/git/" target="_blank">#git</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://steinar.bang.priv.no/tag/gitstoremeta/" target="_blank">#gitstoremeta</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://steinar.bang.priv.no/tag/metadata/" target="_blank">#metadata</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://steinar.bang.priv.no/tag/rcs/" target="_blank">#rcs</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://steinar.bang.priv.no/tag/svn/" target="_blank">#svn</a></p>