Skip to content. | Skip to navigation

Personal tools
Log in
Sections
You are here: Home

jorgenmodin.net - Blog

Pipelines in Kotlin (first steps)

Posted by admin |

I cannot lie, I like pipelines. They make a program easy to understand, and you get functional programming for free. And parallelization. So how do you do it in Kotlin? In Javascript I use the Bluebird library, which has some extra goodies too. But since I am now looking into Kotlin, what's the deal there?

Java itself now has streams, but Kotlin's support for pipelines predate Java's. Many types can be pipelined in Kotlin, but in Java they need to converted to streams. Kotlin's pipelines are eager, unless you convert them to sequences, in which case they match the lazy evaluation of Java streams. Here is a Kotlin program that seems to work:

package se.webworks.pipes

fun main(args: Array<String>) {
    val myList = listOf("a1", "a2", "b1", "c2", "c1")

    myList
            .filter({ s -> s.startsWith("c") })
            .map( { emBiggen(it) })
            .sorted()
            .forEach({s-> println(s) })
}

fun emBiggen(thing: String): String{

    return thing.toUpperCase()
}

 

"it" is the default variable where things end up if you do not specify. I like that. Very Hypertalk-y.

I learnt it all here http://www.baeldung.com/java-8-stream-vs-kotlin.

What I am wondering now, is if you can curry functions, so that you can pre-configure them with some parameters already in the pipeline. Maybe just removing the brackets and have a function return a function will do the trick?

We will see…

Why pipelines then? As Walther Bright, creator of the D language wrote:

With these thoughts in mind, I look back at all my failures at reusable code and notice something else: It looks nothing at all like: source → algorithm → sink. In fact, it looks like a bunch of nested loops. The source data enters at the top, and gets swirled around and around in ever smaller and tighter loops, and leaves via the sink in the center of that maelstrom.

Read more: Link - Component Programming in D | Dr Dobb's

 

 

 

Dec 03, 2017 10:20

Setting up Kotlin on Ubuntu 17.10

Posted by admin |

Summary:

  • Use Oracle's Java, OpenJDK does not work with IntelliJ on Ubuntu 17.10
  • Install Maven
  • Start project in IntelliJ via Maven

Longer explanation

First install maven. It is in the Ubuntu repositories, so easy step.

sudo apt install maven

Download IntelliJ. Kotlin is included in IntelliJ, so no need to download it separately. You need to install the JDK from Oracle, because IntelliJ doesn't understand Openjdk (or the OpenJDK JDK is incomplete) on Ubuntu 17.10 at this point in time.

Place it somewhere and then tell IntelliJ where it is, you can download it and e.g. move it to /usr/local/lib/jvm.

 

Screenshot from 2017 12 03 18 11 52
Click to view full-size image…
Size: 161.0 kB

Dec 03, 2017 06:20

Getting identical Argon2i hashes with C reference, Rust, Kotlin/Java, python/libsodium & javascript

Posted by admin |

How do you know the Argon2i library you are using is giving you the correct hash? One way to boost confidence is to see if independent implementations yield the same result.

Below first going through configuration differences, then showing an example of an identical configuration for the five Argon2i implementations, yielding the same hash:

cc894b3e1345fcc3f36c0f9b808021160ec34a97441987ffb7a775bb0c34d5e8

Tested:

  • Web interface to a pure javascript/wasm implementation at http://antelle.net/argon2-browser/, based on a slightly modified version of the reference implementation
  • PyNaCl-1.2.0 for Python3, using libsodium
  • The argon2 command line tool in Ubuntu 17.10, version 0~20161029-1, based on the reference implementation
  • Pure Java implementation by Andreas Gadermaier, version 0.1
  • Rust implementation rust-argon2 by SRU-Systems, version 0.3.0

Different numerical versions of Argon2 yield different results

All above implementations above conform to the hash yielded according to version 0.13 of Argon. However only one of the libraries explicitly state so, rust-argon2. Kudos to SRU systems! You can change the version of Argon2 used in rust-argon2, and with version 0.10 you get this hash (for the same parameters):

ffd0caa5b4df1587f06033e6f1d0060e75742cbad0f7193f4bc97297d3977d76

The Argon2 paper states in its change log for what's new in version 1.3:

• The blocks are XORed with, not overwritten in the second pass and later

It seems to me that being explicit with what version you are using, or even be aware of that there are different version yielding different results, is a pretty big deal. Your password checking or keystretching may otherwise become quite infuriating and stressful.

There is another Rust implementation of Argon2 Argon2rs, that yields the 0.10 hash in the version available at crate.io: 0.2.5 at the time of this writing . There is work underway to conform argon2rs also to the 0.13 standard.

Configuration

The following parameters give the same hash for all five listed implementations. The argon2i version was used, since it is available in all.

Salt

Impl. Name Format
argon2-browser salt string
PyNacl salt byte string b"abc"
argon2 CLI <first argument> string
Kotlin/Java <second arg to hash> "abc".toByteArray
rust-argon2 <second arg to hash_raw> byte string b"abc"

 

Iterations

Impl. Name Format
argon2-browser iterations number
PyNacl OPS_LIMIT integer
argon2 CLI -t string
Kotlin/Java setIterations integer
rust-argon2 time_cost integer

Password

Impl. Name Format
argon2-browser password string
PyNacl password byte string b"abc"
argon2 CLI <STDIN> string
Kotlin/Java <first arg to hash> "abc".toByteArray
rust-argon2 <first arg to hash_raw> byte string b"abc"

For the argon2 CLI:

echo -n 'password' | argon2 […]

Memory

Impl. Name Format
argon2-browser memory number in kibibytes
PyNacl memlimit integer in bytes
argon2 CLI -m string, power of 2 in kibibytes
Kotlin/Java setMemory integer, power of 2 in kibibytes
rust-argon2 mem_cost integer in kibibytes

Memory, as it is called in argon2-browser is called memlimit in pynacl. If you set it to 1024 in argon2-browser, because it is in kibibytes, it should be 1048576 in pynacl, which uses bytes as unit. Google can do the conversion for you.

The argon2 command line tool wants kibibyte powers of 2, so "10" will set it to 2¹⁰ kib which is 1024 kibibytes. Same for Java/Kotlin version.

Octets output length

Impl. Name Format
argon2-browser Hash length number
PyNacl <first argument> integer
argon2 CLI -l string
Kotlin/Java   set to 32 always, maybe?
rust-argon2 hash_length integer

 

Output in hex format

Impl. How
argon2-browser always in hex
PyNacl key.hex()
argon2 CLI -e
Kotlin/Java always in hex
rust-argon2 hexify yourself

In argon2-browser, it is always in hex, in pynacl it is the .hex() method on the result object. In the argon2 it is hex by default but can be changed to raw bytes with the -r flag. In Java/Kotlin version, it is hex.

Examples of an identical configuration of all

Argon2-web screenshot:

Argon2-web
Click to view full-size image…
Size: 34.1 kB

Pure java implemetation called from Kotlin example (thanks to Mikael Ståldal for help on this):

package se.webworks

import at.gadermaier.argon2.Argon2Factory

fun main(args: Array<String>) {
    val password = "masonit".toByteArray()
    val salt = "0123456789ABCDEF".toByteArray()
    val hash = Argon2Factory.create()
            .setIterations(8)
            .setMemory(10)
            .setParallelism(1)
            .hash(password, salt)
    println(hash)
}

 

You can also call the java jar directly with command line arguments:

echo -n "masonit" | java -jar argon2-0.1.jar 0123456789ABCDEF -i -m 10 -p 1 -t 8

Rust example with rust-argon2:

extern crate argon2;
extern crate hex;

use argon2::{Config, ThreadMode, Variant, Version};
fn main() {

let password = b"masonit";
let salt = b"0123456789ABCDEF";
let config = Config {
    variant: Variant::Argon2i,
    version: Version::Version13,
    mem_cost: 1024,
    time_cost: 8,
    lanes: 1,
    thread_mode: ThreadMode::Parallel,
    secret: &[],
    ad: &[],
    hash_length: 32
};

let hash = argon2::hash_raw(password, salt, &config).unwrap();
    let hex_string = hex::encode(hash);
    println!("{}", hex_string);

}

argon2 command line tool example:

echo -n 'masonit' | argon2 0123456789ABCDEF -t 8 -m 10

Python code example:

from nacl import pwhash

password = b'masonit'

kdf = pwhash.argon2i.kdf
salt = b'0123456789ABCDEF'

Alices_key = kdf(32, password, salt,
                 opslimit=8, memlimit=1048576 )
print(Alices_key.hex())
Nov 30, 2017 11:15

Happy to live in the EU?

Posted by admin |

happy
Click to view full-size image…
Size: 52.0 kB

 

http://ec.europa.eu/commfrontoffice/publicopinion/index.cfm/Survey/getSurveyDetail/instruments/SPECIAL/surveyKy/2179

Nov 23, 2017 12:11

Internet, geopolitics & societal organization in light of the reformation

Posted by admin |

This year it is 500 years since Martin Luther affixed his theses to a church door. The reformation, or rather maybe the printing press, changed society and beget wars. What will happen when the Internet changes how we get and disseminate information?

Let's first look at what similarities and differences exist between the effect of the printing press on society and the Internet's effect on society.

With regards to religion, Protestantism supplanted a hierarchical system of clergy, a body of people with relationships between them, with the "leaf nodes" being the common church goers. People lived in kingdoms, principalities and similar, where power was effectively shared between the king or other leader, and the church.

With Lutheranism the king could get rid of the power of the Roman church, and with the direct relationship between scripture and the believer, there was a new concept of identity and self, which both factors may have facilitated the nation state (see Fukuyama: Political Consequences of the Protestant Reformation, Part I , Part II ).

One may even argue that it influenced Catholic France through among other factors the French revolution, and led to Napoleons' Grande Armée which size dwarfed any other armies in Europe, People where energized to fight not for the church, but for other things such as universal ideas and the nation of France. These ideas later turned into the nation state as a war machine, where a certain loyalty with and fealty to the church and a willingness for sacrifice, had been outcompeted by having the same feelings towards the nation state instead.

The printing press led to a more direct relationship between the reader and a rich selection of standardized, mass-produced messages. Radio and television would later severely limit the selection of messages and hence funnel people's thinking into narrower tracks, and in some ways veer closer to a similar grip as that of the medieval Catholic Church, but in the service of the state. Radio may have been instrumental to totalitarianism.

The printing press made it possible to mass produce a wide variety of messages, still some works prevailed and got a much bigger audience. That is, even though the medium itself allowed a wide diversity of messages, people voluntarily limited themselves to a a more narrow set. Why did they do that? Two possibilities arise:

  • Some works were of much higher quality
  • A network effect existed, i.e. there was value in reading the same thing as others

Let's here exclude texts that were of immediate practical use such as on engineering or agriculture, and in doing so I believe the second factor becomes of most importance. Not so that the first factor, the quality of the work is unimportant, but rather that its quality should be viewed in its normative power for how people interact, which is a combination of the accessibility and the power of its ideas.

A thorough examination of the impact of Protestant ideas is beyond the scope of this text, mainly because I do not have the overview of it, but let's just settle for that the ideas did have an enormous impact and not bother too much about in what ways. Then we are left with the analysis that these ideas shaped the interactions between people in new patterns. In Fukuyama's texts he makes a difference between Lutheranism which tended to convert whole countries and principalities top-down, and Calvinism which spread more inconspicuously throughout the fabric of society.

Let's now look at the Internet. Just as with the printing press, a discourse gets delivered to your very own eyes, and you are required to believe in the text rather than to be commanded or prodded into believing in something, as was more the ways of the Catholic Church. And maybe it is this impossibility of commanding somebody to comply to the tenets of a text, where the concept of predestination is conceived.

Why would one embrace a discourse from a book or from the Internet?

  1. It confirms and completes already held views
  2. It promises success, as can be witnessed by those already holding the views

The Internet provides both reasons. The first one is prevalent in spades, and is what we refer to as the "filter bubble". The second one is seen in crypto currencies, which are creatures wholly dependent on the Internet and impossible to fathom without it, where a coin's success is completely dependent on other people's valuation of it.

The first reason, to confirm and complete already held views, can be a bit dangerous insofar that it may lead people into believing that they are more powerful in the world than they are, and that their peers are people that are actually surrounding them, instead of being spread out on the Internet. And of course it can nurture parochial views that are incompatible with interaction with other people holding different views. Some of these bubble communities — such as the flat earthers — have their fair share of people in them who cannot function in society in a productive way, which implies they are not locii of tremendous world-changing power.

The big effect of the Internet bubbles right now is in disinformation, that people do not know what facts to believe in. Partly this is due to some of the facts previously held as true were merely articles of belief meant to function as a glue in society.

When this glue comes undone, we may veer more towards anarchy or libertarianism. Crypto currencies can here be seen as a part of the new glue to hold society together, but that power also means that they may become brutal forces of change. The danger of disinformation on the other hand lies mainly in having difficulty to respond to nation states less affected by the chaos of the Internet, mainly Russia and to some extent China. However those countries may suffer much more gravely once the effects of the Internet hits them. As the entrepreneur Naval Ravikant said, China's monetary policy (to shut out crypto currencies) now boils down to its firewall policies.

The world completely went off the gold standard in the 1970s, and the flaws of "fiat" currencies (as currencies not backed by gold or similar are called) have been amplified by the super speed of computers, where the life cycle is accelerated by operations and communications close to the speed of light and the speed of logic gates. In fact, the fiat monetary system and its accompanying belief system can be seen as the original computer & network-driven filter bubble.

Fiat currencies are connected to the nation state, since they get their value from being the unit for paying taxes and are defended by the monopoly of violence of the nation state. If we assume the nation state will be weakened by the Internet and by crypto currencies, fiat currencies will also be weakened (directly so by crypto currencies obviously). Incidentally the Euro fiat currency has put itself into a strange predicament by being used in a geographic area without the rigor of the nation state.

Timothy Snyder has warned about the dangers of not having a functioning state. Mortality in such areas tend to go up and sometimes even surpass communist states' death tolls. So it seems we would very much prefer to have:

  • Functioning states, in tune with the new technology
  • An orderly way to get there

 

 

 

 

Nov 05, 2017 06:45

How to get DNS working on a Ubuntu 16.04 machine with a bridged interface

Posted by admin |

Summary: The solution is to disable DNSMasq in /etc/NetworkManager/NetworkManager.conf .

It's no fun when it's not you configuring things wrong, but a bug. Reason is that you haven't learned anything, just made up incorrect reasons, until you find the bug.

A curious thing happened today, a Ubuntu 16.04 machine running a couple of KVM guest machines lost it's connection to the Internet, but the guest machines kept theirs!

After a closer look the host machine still had an Internet connection, but its DNS did not work. I will not bore you with the meandering trouble-shooting path, but in short this is a conflict between /etc/network/interfaces and the NetworkManager. These seem to be different competing systems for configuring your network, and sometimes they do not agree.

This time the conflict was about /etc/resolv.conf, which nowadays is handled by other processes, so if you manually write in it, that will be over written.

In this case bridge-utils (or possible a program triggered by it) wants to write whatever DNS settings it's configured with in /etc/network/interfaces , to /etc/resolv.conf. At the same time NetworkManager wants to tell /etc/resolv.conf that it has a DNSMasq DNSproxy running on localhost. And that does not work, maybe because DNSMasq only reponds to NetworkManager,or it's not there, or it's erroneously configured or whatever. /etc/resolv.conf gets clobbered with 127.0.0.1 and that's it.

The solution is to disable DNSMasq in /etc/NetworkManager/NetworkManager.conf . Then it works and /etc/resolv.conf takes its information from /etc/network/interfaces .

Bug #1384394 “/etc/network/interfaces: “dns-nameservers” entries...” : Bugs : dnsmasq package : Ubuntu

I suspect NetworkManager is only installed on desktop systems, and this may explain why it was hard to find info on the conflict

Oct 25, 2017 11:30

How to re-enable tap to click in Ubuntu 17.10

Posted by admin |

After the upgrade, tap-to-click stopped working with the touchpad. Couldn't find the setting in the, umm, settings applet, and also not in Gnome tweak tools. A solution was found by installing dconf-editor and navigating as indicated below:

 

Selection 001
Click to view full-size image…
Size: 45.5 kB

Oct 21, 2017 11:50

How to easily type weird characters on Ubuntu Linux 17.04

Posted by admin |

To get …  type Caps Lock , release and then ..

Well, not now. When you've done the following steps:

1) You need to install gnome-tweak-tool

sudo apt install gnome-tweak-tool

2) Go to the "typing" section

3) Click at "Position of Compose Key"

4) Select a key. The "levels" refer to diffferent combinations of shift, Alt Gr and so on. Caps Lock however  is a good choice :)

Now you can type ellipsis with <compose>, then ..

and em dash with <compose> then ---

And so on!

It's a bit hard to find which combinations work. Some of these work: GtkComposeTable - Community Help Wiki

Oct 18, 2017 11:30

When screen info is missing in Ubuntu, install lxrandr

Posted by admin |

At least it is worth trying.

gnome-control-center refused to show any info on connected monitors.

Selection 245
Click to view full-size image…
Size: 3.2 kB

 

So I installed lxrandr, the monitor configuration utility from the LXDE project. It worked like a charm.

Selection 244
Click to view full-size image…
Size: 15.2 kB

It also worked for multiple monitors.

Oct 16, 2017 09:15

3 reasons for independence/insulation/secession & how to handle them

Posted by admin |

Catalonia wants independence, the UK has voted for Brexit and the United States under Trump wants to break up free trade agreements and "bring jobs back home". I believe one can see three different reasons for wanting to have independence, or wanting to secede or insulate oneself.

1) Being oppressed - if you cannot speak your language or cannot conduct business and other societal functions in line with your old traditions and customs, or if your religion is suppressed, then you may qualify for being oppressed.

The solution here is to liberalise society so that people can speak and worship what they want. If EU regulations lack in this respect, they must be revisited and improved, so that freedom prevails. A trickier take on this is if you are a minority within an oppressed area. Gaining independence may not necessarily be good for you. There are also demographic trends over time, such as in Singapore and Kosovo.

2) A large part of the population feel they are losing out to free trade and freedom of movement. This is what fueled Brexit and got Trump elected. The solution here is to keep the majority of the population happy.

If the discontent is due to immigration, you may eventually need to dial back on it. Free trade is so important that we do not want that jeopardized in the process! You will need to bring people into a structure that gives them leadership and hope. The best way to do that is to deregulate and make entrepreneurship do its work, and lots of training. If that is not an option, you may need to make a left turn and get people into jobs that way. But it cannot be ignored when it tips 50% discontent, for obvious democratic reasons.

3) A rich region does not want to pay for the poorer parts. This has been discussed in connection with Catalonia and also in Northern Italy.

The solution is to allow regions to "divorce", but they will have to pay alimony. In the EU the EU could set that sum. We do not want too many regions to gain independence, since it would risk a crumbling of the EU into nation states, which would then be somewhat of vassal states of the U.S., Russia and China. We do not want the development since it may be unstable, and wars may erupt depending on the power dynamics of the great powers. Russia is also not strong enough to play this in the long run, so the nation state scenario is inherently unstable.

Oct 04, 2017 10:50