I'm trying to figure out how to make the source code of a system readable, readable meaning that you can understand the system on a high level, and also its constituent parts, from the source code.

It seems to me, and I am just starting thinking about this, that there things (patterns, frameworks, data schemas) that are invariant, that is they don't change over time as the system runs.  I'm sure the word invariant has other definitions, but for this text that was invariant means. These invariant things are the things you can communicate to the developer/maintainer.

So it would seem that you should be upfront with your invariant stuff. Like in a central file or as imports to it. And if you have stuff that is changing and amorphous, can it be abstracted into, or at least abstractly described, as something invariant? Then do that.

It seems to me that the more of these invariant stuff you can get into your system and systems description, the easier it the system will be to understand.

However the invariants, such as e.g. a framework's modus operandi, should not lead to convoluted solutions.

So the rule might be:

Use as many invariants as you can in your system, but not to the point of convoluted constructs.

Could be a two step process: Find and describe the invariants you already have, and see if you can refactor code into invariants.

The invariants can work on many levels: Using a language that discourages TIMTOWDI, using frameworks (iff they fit the problem) and taking time to create well defined abstractions for as many parts of the code as possible. Abstractions here means in the direction of readability and understandability.

Another question then arises, what abstractions are easy to undertstand? In user interface design one measure of the complexity of a GUI is the number of objects on the screen and how many alignment lines there are. Fewer objects and aligned objects lead to lower complexity. I wonder how could one measure the complexity of abstractions in source code? Metrics could include

• how many inputs
• if there is mutable state in there somewhere
• if there are side effects
• how many other files the code interacts with
• the cyclomatic complexity

Abstractions already familiar to the developer are easier for him to use, and re-using known abstractions with some magical twist may work too. it also means it may be a good idea to have a toolbox of powerful abstractios that covers most of the things needed to be done.

Sometimes there is a like precipice when interactions between even simple components get too complex to understand. In which case the sub system needs to be "black-boxed" and not leak into the environment.

If you compile bitcoind version 0.11.0 with the --disable-wallet flag set, certain commands go missing. That is only natural since commands that need a wallet won't work without one.

Curiosly enough the "generate" command in regtest mode, a command that lets you build up on the number of blocks you pretend that you've mined, also disappears.

Well now as I am writing this I realised that it may not be so strange at all. It has to do with that the mining fees need to be paid to an address, and without a wallet, no address. However a bitcoind server in regtest mode can still be a part of a bitcoin network, so one wouldn't necessarily have the the wallet on the server doing the mining.

Well, now I have some 15'000 bitcoins. Not real ones unfortunately.

Update 2015-08-31: Something is on its way: 'virtualization support' - MARC

While searching I noticed there was not much activity in the field of using OpenBSD as a host operating system, with some solutions such as VirtualBox and Vmware not supported at all.

Theo de Raadt, BDFL of OpenBSD, has said the following about x86 virtualization:

x86 virtualization is about basically placing another nearly full kernel, full of new bugs, on top of a nasty x86 architecture which barely has correct page protection

So the support for x86 virtualization in OpenBSD is less than enthusiastic.

Read more: Link - 'Re: About Xen: maybe a reiterative question but ..' - MARC

Some info is available though:

Running a Linux VM on OpenBSD

Install and Run Windows Within QEMU Virtualization on OpenBSD or Linux

In one way we may treat the massive debts in the world lightly: We are still rich in large parts of the world, and all the stuff we have will not disappear overnight. However given the absurd size of global debt there will be deep - potentially vicious - conflicts about who should pay how much to whom. When all those agreements on debt turn out to potentially be worth nothing, we will not trust each other that much onwards with credit.

If trust is gone, why would you deliver a car to the other side of the world, not knowing if anyone will pay, or if what you were paid in is actually money? Bitcoin and other trustless crypto currencies may be the only means of international payment remaining.

Disclaimer: I have no idea of what I am doing.

I have a WD My Cloud with an ARM processor running on 4.x of WD's OS, which is  a Debian with large page sizes. From my 32-bit i386 normal page sized Linux I just tried this:

GOOS=linux GOARCH=arm GOARM=7 CGO_ENABLED=1 go build -ldflags "-R 65536" hello.go

And that runs on the WD device. Not sure if it just dumb luck, maybe it won'y work for bigger programs.

Update: I have now tried to compile a much bigger program and it seems it worked fine, but I got

fatal error: runtime: cannot map pages in arena address space

Which if I undertand golang correctly, is an out of memory error. I never thought of how much RAM my little NAS has. It turns out it has 277MB of which 45MB is free. That is a bit on the low side for the rather huge project I was trying to run. I had to drop the CGO_ENABLED flag but I think that is only needed if there is C code in the project. I have an idea of putting a USB stick into the USB3 port of the NAS and use that as swap to increase the memory available.

I got the flags and switches from here, but I think that guy also built a special golang compiler:

panic on arm cross compile - Google Groups

Cross compilation just got a whole lot better in Go 1.5 | Dave Cheney

Untested by me, but looks promising. I wonder how many people use it?

 

Obnam can also access the live data over SFTP, instead of via the local filesystem. This means you can run Obnam on, say, your desktop machine to backup your server, or on your laptop to backup your phone (assuming you can get an SSH server installed on your phone). Sometimes it is not possible to install Obnam on the machine where the live data resides, and then it is useful to do a pull backup instead: you run Obnam on a different machine, and read the live data over the SFTP protocol.

Read more: Link - Backups with Obnam

Leaving the privacy issues aside for the moment, relying on snooping to fight terrorrism is a very dangerous game. Once it stops working, society has no good way of defending itself. It is better to go down the route of physical security.

Amendment 2017-04-05: This goes also for using the financial system as leverage: When you are not callling the shots anymore, and your enemies are routing around your financial system, what are you gonna do?

The UK government is considering to outlaw all communication that cannot be eavesdropped.

Being able to eavesdrop on all communication is a very cheap way to fight terrorism. It creates leverage. But leverage can also be dangerous, when it suddenly does not work anymore, such as if the government loses its ability to eavesdrop due to a new piece of software, algorithm or usage pattern. Much more focus should be given to hardening the physical defenses of critical infrastructure and better policing. These methods will not fail catastrophically, while snooping may and probably will do so in the future.

It is unfortunate that we may need to expend a lot more money on physical security and travelling encumbrances, but as we get more freedom in cyberspace, other freedoms in the physical world will change.

Talking about this issue earlier this year, David Cameron questioned that if in their country, they should allow a means of communication between people which they cannot read, reported the Daily Star. Answering his question with a no, Cameron had said that being the Prime Minister he will make sure it is a comprehensive piece of legislation that makes sure that they didn't allow terrorist safe spaces to communicate with each other.

Read more: Link - UK to ban WhatsApp under 'snoopers charter' law | Business Standard News

WD My cloud 3 TB network attached storage. Slick web interface.

Lots of connectivity to apps and punching through firewalls, and I have disabled all that. Runs a fairly standard Debian Linux, but with a different page size, and that means you cannot enable the normal Debian repositories. You should be able to hack and downgrade the device to its version 3.x which has a normal page size, but I never got that downgrade to bite.

It has gigabit ethernet, but saturating with a 100Mb transfers with rsync seems to make it veeery sluggish, so not sure if the Gb/s speed is actually attainable.

it has USB 3, but only for connecting to hard disks and similar, not to a computer. It wants to be the USB host. Kind of strange choice, that.

I just wanted a cheap usable NAS server under €200 (2000 SEK, $220).

Ethernet, USB 3.0. Pick one.

I couldn't find one that had both. Or I actually got a WD My Cloud 3TB. And it has USB 3, but only for connecting to hard disks and similar. It wants to be the USB host. It cost me SEK 1500 (€150, including 25% sales tax). Didn't notice that it only had USB host mode until after I bought it.

In the same price range there was a Seagate, but it only had USB 2. In hindsight that would have been better than no USB connectivity to the computer. However I decided to keep the WD device since it has more documentation on care and feeding of the Linux it runs, than what I could find for the Seagate. However it turned out that WD has modified the page size in such a way that you cannot install normal Debian packages. And the workarounds I have found have not worked so far. Ah, well.

OpenSSL has been around for a very long time, but many believe it is too convoluted and ambitious/unfocused to be secure. The OpenBSD people have forked to LibreSSL and Google has forked to BoringSSL.

• LibreSSL is a version of the TLS/crypto stack forked from OpenSSL in 2014, with goals of modernizing the codebase, improving security, and applying best practice development processes.
• boringssl Git repositories - Git at Google

The unveiling of BoringSSL, as the Google fork has been dubbed, means there will be three separate versions of OpenSSL, which is best known for implementing the secure socket layer and transport layer security protocols on an estimated 500,000 websites. Developers of the OpenBSD operating system took the wraps off LibreSSL a few weeks after the surfacing of Heartbleed

Read more: Link - Google unveils independent “fork” of OpenSSL called “BoringSSL” | Ars Technica

CentOS 5.11 32-bit: glibc version 2.5

Ubuntu 10.04 LTS 32-bit: glibc version 2.10

Debian 6 32-bit: glibc version 2.11

CentOS 6.6 32-bit: glibc version 2.12

Debian 7 32-bit: glibc version 2.13

Ubuntu 14.04 LTS 32-bit: glibc version 2.19

...very well it seems in my initial tests. When running a python package in CentOS 5.11 I got:

 undefined symbol: EC_KEY_new_by_curve_name

Now I've found this discussion, not mentioning Red Hat Enterprise Linux 5.11 (which CentOS is based on) by name, but it seems like support was withheld due to copyright worries:

Bug 319901 – missing ec and ecparam commands in openssl package

One workaround seems to be to compile your own OpenSSL libraries.

I then installed CentOS 6.6 and got a similar error message:

RuntimeError: openssl error: error:100AE081:elliptic curve routines:EC_GROUP_new_by_curve_name:unknown group

but I am just looking for a supported distribution with an old glibc that I want to use to get the widest support for a python package that will be made into one file with PyInstall. So off to the next candidate for that. Or I'll compile an OpenSSL for CentOS 5.11 and see what actaully does get installed with PyInstaller.

Untested how-to here for CentOS 6:

Install bitcoind on CentOS 6.5

Latest right now (2015-07-07) seems to be:

openssl-1.0.2c.tar.gz 

Starting with Python 2.7.9, it is included by default with the Python binary installers.

But if you install it is not there in the bin directory! Drama and confusion! Or at least confusion. It can be installed without accessing the network like this (with your built python):

python -m ensurepip

So it is there. And shows up in the bin directory. It just needs to be summoned.

Read more: Link - Installing Python Modules — Python 2.7.10 documentation

One way is to build from source, after having installed prerequisites. Tested by me right now, worked like a charm! If you are going for 32-bit change "x86_64"

in:

yum install gcc gcc-c++.x86_64 compat-gcc-34-c++.x86_64 openssl-devel.x86_64 zlib*.x86_64

to "i386".
 

Read more: Link - centos5 - How should I upgrade Python on CentOS 5.5 (64-bit) - Server Fault

1. Check what is actually sent over the network and what is actually stored on file/in a database

This is your data. Everything else is just crunching that data.

Check so that there isn't something wrong here, because if there is you fall victim to the GIGO (Garbage In, Garbage Out) principle, and it does not matter how well the code itself is written. I once had a problem with Internet Explorer (IE7 I think) not displaying stuff the same way as the other browsers. Turned out after quite some debugging, that it was sent a subtly different version of the web page.

2. If you're looking at something for more than 40 minutes, you're looking in the wrong place

3. There is no such thing as rollback - switch back instead

When rolling back an upgrade: Replace with previous binary structure instead.

Back when I installed and upgraded Plone based systems, there was of course the version control system, but also the ZODB appending database, hundreds of python eggs, some files and the operating system libraries. All changes in these can be rolled back, but did you miss anything? Is something somewhere not pinned properly? If so, how long would it take you to find it? If you just make a binary copy of the old system (to the extent that you can), you can just switch back to that system.

4: Testing must be done as close to the user as possible

It does not matter how good the system is under the hood, unless the last mile to the user is also in perfect shape.

• Default & start from scratch
• Reform society to be agile and economically efficient
• Build trust between people
• Never take loans again

Here's a longer how & why for each one of these:

Default & start from scratch

Being saddled with huge debts is a very bad situation, whether for an individual or family, or for a nation. Over a certain threshold it just becomes impossible. There are examples of countries paying back debts. However a simpler option is to just default. This will also teach creditors to be more careful lending out money in the future.

Reform society to be agile and economically efficient

A society where people can quickly organize and reorganize themselves and their production resources, will have a much smoother ride through times of change. This means low taxes on work, promoting work and division of labor. High taxes kill division of labor since with high taxes it is cheaper to DIY stuff than to outsource it to specialists: For every new person in the chain the tax on wages will take a cut, so with e.g. 50% tax on wages the third person in the chain would receive 25% of what the first person had to earn to pay him. With 30% tax it gets around 50%. So you see how high taxes effectively puts a stop to chains of cooperation.

It must be easy to hire and fire people and to start and scuttle companies. There must be an efficient conflict resolution system. But what if hire and fire creates alienation in society?

Build trust between people

Trust is needed between people to make for a good society. If trust breaks down you will either have anarchy & chaos or a strong leader who oppresses and solves things inefficiently. There are some rather good books about how to start building trust in society by Gene Sharp. Negative income tax or just a guaranteed sum to live from are also interesting ideas. Simple systems are less vulnerable to capriciousness and corruption.

Never take loans again

Loans are probably a bad idea, and should be abolished. This is one of those times when several religions actually got it right. Instead sell equity. With equity in bad times somebody may own what you built but they will not own you. Big difference.

I used tmuxinator and byobu for this. Tmuxinator allows you to start one or more sessions of tmux or byobu (programs that gives you virtual terminals) and for example tile them.

It was hard to predefine setup commands in tmux or byobu directly, but with tmuxinator you can do it in a YAML file. I believe it uses sendkeys instead of relying on tmux' command parameter.

Manage complex tmux sessions easily

(for any nitpicker reading this, I mean what is referred to as panes in tmux and byobu when I write sessions.)

• It should be open source
• No code should be loaded from a server, all code should be local when running
• You should be able to store your passwords on your storage of choice
• Storage should be encrypted safely
• You should be able to update the database from different devices
• Browser integration should be stellar, including generating passwords for you

It seems to me that this can be split in two areas: back-end and front-end. Many projects seem to be good at the back-end stuff, and not so many at the front-end stuff. I wonder, would it be possible to split the work so that one open source project only does the front-end work, i.e. the browser integration, and then make that pluggable?

Something I find quite hard in English is remembering which words end in -ous and which end in -uous.

Especially the latter ones. Here are some examples:

• ambiguous
• arduous
• assiduous
• conspicuous
• contemptuous
• contiguous
• continuous
• disingenuous
• impetuous
• incongruous
• ingenuous
• innocuous
• promiscuous
• sensuous
• strenuous
• superfluous
• tenuous
• tortuous
• vacuous
• virtuous
• voluptuous

Found this list:

The following table lists in alphabetical order the most commonly cited programming fonts.

Font Survey: 42 of the Best Monospaced Programming Fonts - CodeProject

My favourites from looking at the images are:

• Proggy Clean with Slashed Zero
• Terminus
• Liberation Mono