Skip to content. | Skip to navigation

Personal tools
Log in
Sections
You are here: Home

jorgenmodin.net - Blog

YAML examples

Posted by admin |
May 13, 2016 06:27

How to include a multi line string in YAML

Posted by admin |

Put just a pipe character on the first line:

foo: |

Then add the string, indented on the following lines:

foo: |
    bar
    baz
    bletch
    flum

 

May 10, 2016 11:39

Nollräntor och QE kan bli liberalernas största blunder

Posted by admin |

Att liberaler tillåtit negativa räntor, nollräntor och penningtryckning kan vara ett  misstag av historiska proportioner och hotar nu liberalismen över hela världen.

Alla dessa åtgärder förstör kapitalismen och desorienterar väljare. Den förstör också kapitalismens och marknadernas goda namn, eftersom åtgärderna utges vara i samklang med dem.

Men: Utan korrekt prisinformation, desorientering. Just någon slags desorientering kan ligga bakom mycket av de förskjutningar mot extremer som vi ser i väljarkårerna både i USA och i Europa, vilket Peter Santesson tagit upp: Den nya pessimismen — Kvartal. Folk reagerar negativt på desorientering, även om mätarna pekar uppåt.

Vi borde aldrig låtit dessa åtgärder passera. Även i de länder där vi inte är i regeringsställning borde vi protesterat högt och ljutt. Det finns inget liberalt i dessa reformer; de förstör informationsflödet mellan långivare och presumtiva låntagare och de flyttar stora värden från de som arbetat ihop dem till de som råkar befinna sig på en speciell plats i systemet. Åtgärderna förstör för fattiga länder genom att skicka penningflöden in och ur dem med stor godtycklighet . Åtgärderna förstör i väst och framför allt i USA genom att invagga oss i en falsk säkerhet att bara för att vi sitter på de finansiella institutionerna just nu och har en slags hävarm, så kommer det alltid att vara så.

Summa summarum kommer vi nog att se en kraftig vänstervridning, ackompanjerat av populistiska högerrörelser bland länder och samhällsskikt som upplever sig vara på tillbakagång.

May 09, 2016 05:30

Mopidy - an mpd server

Posted by admin |

It seems to have some extra tricks up its sleeve compared to e.g. mpd.

 

Vanilla Mopidy only plays music from your local disk and radio streams. Through extensions, Mopidy can play music from cloud services like Spotify, SoundCloud, and Google Play Music.


Read more: Link - Mopidy

May 09, 2016 04:01

Pipelines as programming method

Posted by admin |

Structuring your code in the shape of pipelines seems to alleviate a a lot of problems. I am using promise chains in javascript in this way as we speak. Or I speak. Or write, rather.

 

With these thoughts in mind, I look back at all my failures at reusable code and notice something else: It looks nothing at all like: source → algorithm → sink. In fact, it looks like a bunch of nested loops. The source data enters at the top, and gets swirled around and around in ever smaller and tighter loops, and leaves via the sink in the center of that maelstrom.


Read more: Link - Component Programming in D | Dr Dobb's

May 09, 2016 01:28

WhatsApp krypterar all trafik, tror man

Posted by admin |

Eff.org har en ranking av vilka meddelande-applikationer som har bäst säkerhet. För att få full pott så måste det vara open source. WhatsApp har nu 6 av 7 poäng, med väldigt genomtänkt end-to-end kryptering enligt EFF, men inte open source så det går ju inte att kolla https://www.eff.org/deeplinks/2016/04/whatsapp-rolls-out-end-end-encryption-its-1bn-users

En annan fråga är ju om det är bra eller dåligt med kryptering. Hur som helst, full pott går till:

https://www.eff.org/secure-messaging-scorecard

Apr 09, 2016 02:33

Backing up a running virtual machine on Ubuntu 15.10

Posted by admin |

 

Update 2016-05-09:

Folding back a snapshot in kvm works in Ubuntu 16.10! The whole chain as described below now works. I have not checked if my 16.10 is running on the hand installed debs or if they've been superceded by 16.10 versions.

Update 2016-04-13:

It seems that the last step, folding back a snapshot in kvm, does not work in Ubuntu 15.10 and indeed has a lot of problems on other systems too. Folding back:

virsh blockcommit guestName vda --active --pivot --wait

fails with:

error: internal error: unable to execute QEMU command 'block-commit': Device 'drive-virtio-disk0' is busy: block device is in use by block job: commit

It works if you stop the guest, but that kind of defeats the purpose since it is not backing up a running machine anymore. One may try to use RedHat instead and see if patches there have been quicker, since the Qemu/Kvm people seem to be aware of the problems.

Summary 2016-04-08: KVM can be used and can do this (but last step does still not work, see above), but the configuration and packages in Ubuntu 15.10 are buggy, so you have to replace stuff, and disable stuff:

 

Install KVM, disable AppArmor,  install by hand updated versions of libvirt-bin (libvirt-bin_1.2.16-2ubuntu11.15.10.4_amd64.deb) and libvirt0 (libvirt0_1.2.16-2ubuntu11.15.10.4_amd64.deb), install qemu guest agent in the guest, open up a communications channel between host and guest by creating a channel device called org.qemu.guest_agent_0 in the configuration of the guest.
 
Read on for more info on how to do  this (you can skip to the KVM heading if you don't want the background).

Backing up a running virtual machine on Ubuntu 15.10 turned out to be not so simple. A lot of the information that Google return is outdated, giving recommendations pertaining to 8 year old Ubuntu systems or claiming that KVM cannot run GUI systems.

Virtualbox

I started out with Virtualbox, and Virtualbox has something called snapshots. As far as I understand, when you take a snapshot, the main vdi file becomes read-only and all changes go into a new snapshot file. Several snapshots can be made and they can reference each other and branch, and be folded back into the main vdi file and into each other. I followed this guide:

Arne's Blog: Backup of running VirtualBox machines:

!/bin/bash
VBOXMANAGE="/usr/bin/VBoxManage -q"

if [ $# != 1 ]
then
    echo "Usage: $0 VBoxName"
    exit
fi

echo "Renaming old snapshot..."
$VBOXMANAGE snapshot "$1" edit previous --name deleteme
echo "Renaming current snapshot..."
$VBOXMANAGE snapshot "$1" edit current --name previous
echo "Taking new snapshot..."
$VBOXMANAGE snapshot "$1" take current
echo "Deleting old snapshot..."
$VBOXMANAGE snapshot "$1" delete deleteme
 

But I ended up with errors and inconsistent snapshots that were corrupt according to Virtualbox. Which is probably my fault somehow. Digging deeper I found a blog post referencing the same script, OSC — Backing up Virtual Machines in VirtualBox  and that guy writes that he has gone back to shutting down the virtual machines and then back them up:

Sadly, for me personally, live VirtualBox snapshots haven’t been a terribly robust backup strategy. I’ve unfortunately seen several snapshots fail. Or, worse, had VirtualBox crash while a snapshot was taking place.

Btrfs

So next solution, why not put the virtual machine on a snapshotting file system, that is guaranteed to copy the vdi file in one fell swoop? This may not be enough, in the sense that there might be state stored in other files or in RAM. But it is worth a shot. One option could be btrfs, which I have been experimenting with and which has a very smooth snapshotting mechanism: You can snapshot any file or director on a btrfs volume. But timely enough, this was posted on Reddit's /r/linuxadmin:

BTRFS for VM images? : linuxadmin

which thread says things like:

Yes. I do not recommend this. I spent more time maintaining the one client I had running VMs on a BTRFS store than I did the fifty or so others I had running VMs on ZFS stores, for roughly a year. The replication is unreliable, the performance is incredibly hit-or-miss

The poster lauds ZFS though. ZFS is supported well in Ubuntu 15.10, but the ZFS snapshotting is not as simple as btrfs. Just using LVM should also be an option.

VM options

So, what about changing the virtual machine from Virtualbox to something else? Three options are:

  • VmWare
  • Xen
  • KVM

VmWare costs money for the snapshotting version. I did however vaguely remember that Linode switched from one VM technology to another recently. A Google search turned up that they switched from Xen to KVM.

KVM

Ok KVM it is. What is available snapshot wise there then?

Well firstly there is a way of freezing the state of a VM, backup and then unfreeze it again Backup of running KVM qcow2 VPS - Server Fault.. Secondly it seems to have the same ability to do snapshots with mutable snapshot files as VirtualBox, but there is more documentation giving examples of what I think I want.

This is the best I have found so far [libvirt-users] Backup a VM (using live external snapshot and blockcommit:

# Create snapshot
virsh snapshot-create-as --domain $VM_NAME snap --diskspec
vda,file=$VM_DIR/"$VM_NAME"-snap.qcow2 --disk-only --atomic
--no-metadata --quiesce

# Copy frozen backing file
cp $VM_DIR/"$VM_NAME".qcow2 $SNAP_FILEPATH

# Blockcommit snapshot back into backing file
virsh blockcommit $VM_NAME vda --active --pivot

# Remove snapshot file
rm $VM_DIR/"$VM_NAME"-snap.qcow2

# Variables should be self-explanatory: 
# - VM_DIR is the directory where the VM are stored
# - VM_NAME is the name of the VM, and its qcow2 file is called
# VM_NAME.qcow2
# - SNAP_FILEPATH is the full path (including name) where the backup
# should be created

This seems to be essentially the same KVM – Live backups with qcow2 | Gonzalo Marcote | Open source, open mind.

Some more info here too: Live-disk-backup-with-active-blockcommit - Libvirt Wiki

The above script will only work if the following criteria are fulfilled:

Download links for amd64 are (to the best of my understanding and what I used):

Install with sudo dpkg -i libvirt*

(assuming you only have those two files in the directory)

Unfortunately, without tuning or disabling AppArmor there is currently no way to flatten the snapshot back into the image, as of the last comment currently at Bug #1517539 “Libvirt KVM can not create snapshot (with qemu-gue...” : Bugs : libvirt package : Ubuntu:

"Strictly speaking, the virsh command that prompted this ticket has been fixed. So, I can now successfully create a live snapshot using the proposed packages. Unfortunately, I have no way to flatten the snapshot back into the base image now."

However after having disabled AppArmor, it works, it seems.

Moving from VirtualBox to KVM

Was disarmingly simple. As long as you follow the up-to-date guides and use the virt-manager GUI, you can actually just use the vdi file directly in KVM. However then you will not get the snapshotting abilities. For that you need the qcow2 disk format. Which you can convert to directly from vdi like this:

qemu-img convert -f vdi -O qcow2 vm.vdi vm.qcow2

 

Apr 08, 2016 02:45 | Comments (2)

Snapshotting running Virtualbox instances...

Posted by admin |

...seems very difficult. I tried the script listed here:

Arne's Blog: Backup of running VirtualBox machines

but I end up with corrupt stuff, including the entire Virtual machine. VBoxManage says it cannot delete the deleteme copy and then it all goes downhill from there.

I am probably doing something wrong, but I am now looking at KVM or Xen to see if they can do snapshots. This is a lot trickier than I thought. Maybe I will need to settle for stopping the Virtualbox vm, back it up and then start it again, or somehow back up its entire state from within itself. But that does not feel right.

I'm also looking into using a snapshotting file system such as btrfs for snapshotting VM state, but btrfs has a lot of problems in this context according to this thread:

BTRFS for VM images? : linuxadmin

Apr 04, 2016 11:55
Ads