Tools I built to build this blog

How websites work (very very briefly)

A website is a collection of code that is hosted on a server. If you ask a server for a file (and it is a public file and server) the server "serves" you the file. Many servers are running a large program like WordPress which generates HTML code for each request. It has a database of entries (text you wrote as you compiled your blog) and a server program that populates HTML pages with those entries as you request them. This gives the site a lot of flexibility in showing some pages to some users but not to others, to regenerate pages in real time in response to changes in a database, and to streamline the process of generating new pages for new blog entries. It's also slow, real slow. Every time a user requests a page, the server has to figure out what the links in the menus should be, where the body text needs to fit into an HTML template, and, depending on the site, a near limitless number of other things. Content management systems try to optimize for some of these special cases but they are still slow.

The faster way is a static website, a site where all pages are files on the server and the task the server needs to complete is "here's the page you wanted." This strategy works well when the website is only updated at discrete times, ie when I write a new post. If the site has to look up data to generate something for the user, this does not work so well. Fortunately, my site is static. I write articles, my server sends you those articles, but at no point does the server generate an article. I do that on my home computer.

It might be misleading to think of this as a static site. It clearly does some interactive things. However, all the interactive elements are programs written in javascript. My server does not compute anything for you. My server gives you the code so that your browser can compute things for you. This is why I do not have to worry about data validation for user inputs. No one can infect my server through those inputs because nothing in those inputs is sent to the server. Obviously it would be a better utility if the data validation was more developed but because it is a static site, this is a quality issue not a security issue.

How it is done

First you need an a text file with stuff that is easy for a computer to read. I named all my article files ".article" so they would be easy to find as my script searched through folders. If I were rewriting this today, I would format these files as JSON objects or something similar. However, this format works just fine.

example.article

##Head

keywords: FIRE,ODE,Calculus

author:MC Byington, MCB

date: 2019-06-07

title: Saving time until financial independence has a simple closed-form solution

short_title: Saving time to FI

series_name: FIRE

series_order:1

ref_title: saving-to-FI

description: Derivation of a explicit formula for the saving time until financial independence.

tagimage: tagimage.jpg

tagmini: tagimage_mini.jpg

tagimage_source: https://www.pexels.com/photo/silver-and-gold-coins-128867/

tagimage_author: Pixabay

tagimage_retrieved: 2020-02-05

##/Head

##NewCSS

##/NewCSS

##Variable-table

<tr>

<th><eq> m </eq> </th> <th>Total invested capital </th> <th></th>

</tr>

<more variable table entries

##/Variable-table

##Body

<span class="quote"> The three most harmful addictions are heroin, carbohydrates,

and a monthly salary. </span> <span class="attrib"> - Nassim Taleb in The Bed of Procrustes </span>

</p></div>

<p>The acronym FIRE, financial independence retire early, is the name of a small movement .....</p>

##/Body

Second you need an HTML template. Every page on this site is produced from the same template.

template.txt

<!DOCTYPE html>

<head>

window.dataLayer = window.dataLayer || [];

function gtag(){dataLayer.push(arguments);}

gtag('js', new Date());

gtag('config', 'UA-143422817-1');

</script>

<title>Domestic-engineering: ##shorts:title## </title>

##NEWCSS##

##meta##

##mathjax##

</head>

<body>

</a>

</div>

##headlinks##

</div>

##sidenav###

</div>

##variable-key##

<h1>##shorts:title##</h1>

##headimg##

##body##

</div>

</div>

<a href="https://twitter.com/domesticengine7?ref_src=twsrc%5Etfw" class="twitter-follow-button"

data-size="large" data-show-count="false">Follow @domesticengine7</a>

</div>

<p></p> <!--- I wrote this before I knew this was poor form in HTML.

It was my first website and first attempt at HTML, I know better now :) --->

</body>

</html>

That is my website, all 70-something pages of this writing. They all fit that template. To make that template a useful piece of HTML, I have to replace some fields there.

replace codes in template.txt

##shorts:author## : author field from article file

##shorts:content## : description field from article file

##shorts:title## : title field from article file

##basepath## : as many "../" as I need to reach the home directory from the current folder

##NEWCSS## : the NEWCSS field from the article file, contains any CSS unique or semiunique to that page

I use this a lot with articles that have java script utilities

##meta## : a metatag scripts adds fields so that facebook/twitter/etc will make my links pretty

##mathjax## : depending on whether I am compiling to my localhost or the main server, I use different mathjax code here

##headlinks## : output from a script which builds links to the head bar just below the site logo

headlinks map to the page with the same series_name and series_order==1

##sidenav## : output from a script which builds side navigation links at the left of the screen

the order is taken from the series_order field

##variable-key## : builds a variable key table which responds to the click of a button

(useful for the heavy math articles)

##headimg## : inserts the image at the top of the article with photo credit

##body## : inserts the bulk of my article

Not all pages have a ##variable-key## or ##NEWCSS## field. In those cases, the replace key is replaced with an empty string.

The compiler script has to do several other functions beyond replace those keystrings. It reads the article looking for \cite{ref1,ref2} and a .bib file in the same folder. If it finds both, it reads the bib file and composes a references section exactly as a LaTeX compiler would. My compiler is set up to produce a reference format similar to the IEEE standard.

When compiling for the server, the code ignores files which end in "-draft.article" and produces no links to articles which end "-hidden.article". This gives me some flexibility working on drafts before publication. Articles start as -drafts; as I start soliciting feedback from friends, they change to -hidden so I can send out links.

There are several formatting issues the script handles next. For instance every paragraph which begins <p class='caption'> is replaced with <p class='caption'><span class='caption'>[Caption]</span>. This keeps my figure formatting consistent. If I wanted to, I could use the script to number the figures and distinguish between figure and table captions. That may be a future project.

Next, the script opens Inkscape and converts all text to paths before uploading them to the server. This keeps the figures looking nice on a wider variety of browsers. For example, my father's browser is set to display all text in a huge font regardless of the CSS. This causes the labels in my plots to over run each other but it is easily fixed by converting all those text pieces to paths.

Finally, if compiling for the server, the script runs an rsync command to push my changes up.