I noticed that a thread in a python application that I am debugging, did not turn up in the trace print. Turns out there is a special settrace call you make from the threading package. I linked below to python 3 but it works in python 2.7 as well.
threading.settrace(thread_traceit)
17.1. threading — Thread-based parallelism — Python 3.4.3 documentation
I haven't tried this so I do not know if it is in any way practical, but wouldn't it be possible to classify texts as being in one language? If so, you could filter out any characters or glyph combinations that aren't native to that language.
It would be a bit like how markup languages such as Markdown, reStructuredText, Creole and others have largely supplanted HTML for creating rich text documents on Wikipedia, GitHub, in python documentation and many other places.
Its pretty easy to have two different Unicode strings display identical output - and that can cause a whole host of problems. For instance, many family friendly sites may ban foul language from user comments, but its trivial to come up with Unicode equivalent strings that bypass any blacklist of obscene language.
Read more: Link - Unicode is Kind of Insane
I'm testing a python server from a python test script. The server reads a config file on startup and I want to test that it actually works. So I want the script to test the server as if it were run from the command line. When run from the command line, the server can be terminated by typing Ctrl-c.
The server starts a child process from inside itself. Problem is, if I start the server with subprocess, and then send the kill/terminate signal to it via subprocess, the server's child process keeps running.
The trick is to assign a new process group to the server, separate from the testing script. In this way the os module can issue a "killpg" command that takes care of terminating the server and its child processes, without killing the testing script.
import subprocess import os import signal p=subprocess.Popen(your_command, preexec_fn=os.setsid, shell=True) os.killpg(os.getpgid(p.pid), signal.SIGTERM)
Found this solution on Stackoverflow:
To handle the general problem:
p=subprocess.Popen(your_command, preexec_fn=os.setsid) os.killpg(os.getpgid(p.pid), 15)
setsid will run the program in a new session, thus assigning a new process group to it and its children. calling os.killpg on it thus won't bring down your own python process also.
Read more: Link - unix - Killing a subprocess including its children from python - Stack Overflow
It has been common to assume that man's lot in the world is getting better and better, and that one of the things contributing to this, is less and less death caused by war.
However if you note and measure that the distribution of deaths from war through history is seriously fat-tailed (which means that very rarely occuring wars do kill a whole lot of people), then almost all measurements you will take on the way will be below the true mean. And we may in fact be up for serious mayhem. This is presented in a paper (linked at the end of this post) by Nassim Nicholas Taleb and Pasquale Cirillo, and blogged about by Mark Buchanan here:
Violent warfare is on the wane, right? — Bull Market — Medium -
For example, it turns out that, for a process following this statistical pattern, one should expect fully 96% of all observations to fall below the true mean of the process. This brings home just how non-Gaussian and non-normal this process is. We’re used to thinking that, if we observe instances from some random process, we ought to (very crudely) see events about half above and half below the mean. Instead, in this process, one should expect that almost all observations will be below, and even far below, the actual mean. We almost always see fewer wars than we, in a sense, should.
Original paper: http://www.fooledbyrandomness.com/violence.pdf
It seems that more and more devices interface with Android phones and tablets via the headset jack. As examples here is a brand new geiger counter: http://www.kjell.com/sortiment/telefoni-kommunikation/mobiltelefon-tillbehor/gadgets/geiger-och-uv-matare/geigermatare-for-mobilen-p96153#ProductDetailedInformation
And here is the new credit card processing unit for iZettle (the older one uses Bluetooth). https://www.izettle.com/se/card-readers
I have used the IZettle bluetooth device with different devices and it just doesn't work on all devices, due to as far as i understand different bluetooth stacks.
I recently tried to interface to an Arduino with a USB library through an OTG . It worked on one phone but not on the other.
My guess is why this is happening is that the implementations for Bluetooth and USB support on Android are too buggy or at least diverging in standards.
We do know however that people have standards for music and sound quality, which the manufacturers must abide too or people won't use the product at all. So we are sometimes back to analog modem signals it seems.
Kind of obvious, but it means you cannot just clip them around a normal power cable, such as are used to power household devices that plug into wall sockets.
The primary winding of the CT is the wire carrying the current you want to measure. If you clip your CT around a two or three core cable that has wires carrying the same current but in opposite directions, the magnetic fields created by the two wires are equal and opposite and will cancel each other. Your CT will have no output
Read more: Link - CT sensors - Introduction | OpenEnergyMonitor
Including kind of compatible with Arduino. It obviously can do WiFi, but people have now hacked so that you do not need a microcontroller or computer to control it. It can do that by itself, including accepting a lot of Arduino code. ESP8266 Community Forum View topic - Arduino IDE released for ESP8266
This board exposes all the pins, including serial and PWM MOD-WIFI-ESP8266-DEV - Open Source Hardware Board
Lua based firmware nodemcu/nodemcu-firmware
Read more: Link - ESP8266: This $5 Microcontroller with Wi-Fi is now Arduino-Compatible | Make:
Summary: LightBlue Bean by Punch Through Design seems the most promising.
All cards untested by me at this point in time.
It seems to me that a good use case for the Arduino is as a very low power device that just wakes up briefly, to send some sensor data to a server. Sensor data could be different air quality measurements such as humidity, or certain gases. As long as the bluetooth can connect relatively quickly on wake-up, you ought to be able to make sensoring arduinos that can last for years on a battery. So you could place them in places that are difficult to reach with electrical power.
Here is what I have found after a brief search, and as I said I have not tested any of these:
Bluetooth Bee - Standalone (built-in Arduino) - priced at $23.50 when I read the link, needs a shield in order to be programmed, it seems
RF Digital RFduino RFD22102 - Micro Center - at $21, needs an additional USB shield in order to be programmed
BLEduino: Bluetooth 4.0 Made Easy has a USB port, which might make it less power efficient. Does not seem to be publically available to buy just yet.
Bluz: A cloud-connected, Bluetooth LE development kit by Ben Harris — Kickstarter
At least according to specs, this seems to be what I am looking for: LightBlue Bean – Punch Through Design. It says:
The Bean is the first Arduino-compatible microcontroller board that's wirelessly programmable over Bluetooth Low Energy.
It costs $30 though, so a bit more pricey that the options above. I had to get deep into Google's search results to find LightBlue Bean.
I wonder why the other ones cannot just be programmed on serial pins? Have I missed something?
Hertz the WAV recorder, Record That Note and PCM Recorder had the best recording quality.
The 6 free apps below for Android claim to record high quality audio in uncompressed form such as WAV or PCM. Apps only briefly tested by me at this point in time, with an LG G2 with its factory headset, and me counting to twenty. Lots of white noise in all recordings and no possibility to adjust recording levels.
The apps where found through this useful page: 10 Best Apps for Uncompressed Audio (android) | AppCrawlr
A better microphone is the obvious upgrade from here. I wonder if the noise comes from the phone or the mic+wire. Furthermore, if the noise is generated by atmospheric noise, would it be possible to mix in a real time noise source in anti phase? Would a low impedance output amplifier help between the mic and phone?
A lightly-loaded model B with keyboard = 1.89 W -> daily 45.36 Wh
A lightly-loaded model B+ with keyboard = 1.21 W -> daily 29.04 Wh
B+ with LAN/USB chip off (no i/o except GPIO) = 0.76 W -> daily 18.24 Wh B+ shut down = 0.26 W -> daily 6.24 Wh
The B+ really offers huge improvements in the power circuitry. Wow!
For an Arduino Uno it seems to be around 230 mW at idle, and putting it to sleep helps a bit mut not that much since the USB is still running it seems, so about a 25% saving only.
It seems that a Raspberry pi B+ can compete with an Arduino Uno in idle in terms of power consumption. You've got to get one of the Arduinos without USB to get to the real low power stuff.
How Much Less Power does the Raspberry Pi B+ use than the old model B? » RasPi.TV
Untested by me, but could potentially cut down on the number of components.
Controlling a hobby servomotor is relatively easy. It requires a single digital pulse-width modulation (PWM) signal at 50Hz, which means one pulse every 20ms. The width of the “high” pulse sets the absolute rotation angle of the servo.
Read more: Link - Smartphone Servo | Make:
To Arduino:
ESP8266: This $5 Microcontroller with Wi-Fi is now Arduino-Compatible | Make:
Javascript - Espruino - JavaScript for Microcontrollers
A Kickstarter project. This seem to be aimed at the Raspberry Pi market. A bit worse specs than the Raspberry Pi 2 but similar to the Raspberry Pi B+. Circuitry handling batteries is a boon, and so is built in bluetooth and Wifi .
Differentiating Chip from Beagle is its built-in WiFi, Bluetooth, and the ease in which it can be made portable, thanks to circuitry that handles battery operation.
Read more: Link - The World's First $9 Computer Coming to Kickstarter | Make:
Tried and it does work:
env PULSE_LATENCY_MSEC=30 audacity
Read more: Link - Linux Issues - Audacity Wiki
A good discussion why computer networks have problems staying reliable
Some choice quotes:
The 90-second network partition caused fileservers using Pacemaker and DRBD for HA failover to declare each other dead, and to issue STONITH (Shoot The Other Node In The Head) messages to one another. The network partition delayed delivery of those messages, causing some fileserver pairs to believe they were both active. When the network recovered, both nodes shot each other at the same time. With both nodes dead, files belonging to the pair were unavailable.
--
From what we can gather informally, all the major managed hosting providers experience regular network failures. One company running 100-200 nodes on a major hosting provider reported that in a 90-day period the provider’s network went through five distinct periods of partitions.
--
Large-scale virtualized environments are notorious for transient latency, dropped packets, and full-blown network partitions, often affecting a particular software version or availability zone. Sometimes the failures occur between specific subsections of the provider’s datacenter, revealing planes of cleavage in the underlying hardware topology
--
Marc Donges and Michael Chan bring us a thrilling report of the popular Broadcom BCM5709 chipset abruptly dropping inbound but not outbound packets to a machine. Because the NIC dropped inbound packets, the node was unable to service requests. However, because it could still send heartbeats to its hot spare via keepalived, the spare considered the primary alive and refused to take over. The service was unavailable for five hours and did not recover without a reboot.
Old HP server featuring last Dual Pentium (architecture wise actually the first Core 2 duo):
This machine benchmarks at 57471.3 pystones/second
Laptop with i5-4200u 1.6GHz with turbo to 2.6GHz (no idea if it it turbo'd):
This machine benchmarks at 185771 pystones/second
WD My Cloud 4GB, running Debian 7:
This machine benchmarks at 10869.6 pystones/second
Raspberry pi B+:
This machine benchmarks at 3831.42 pystones/second
So in this particular benchmark, my laptop is about 3 times as fast as my old server. And the Pi is about 15 times slower than my server and 48 times slower than my laptop.
The WD My Cloud is 2-3 times faster than the Raspberry Pi B+, 6 times slower than the server and 18 times slower than my laptop.
Update: I just upgraded my laptop from Ubuntu 14.04 to 15.05, and now it does around 207'000 pystones/second:
This machine benchmarks at 207550 pystones/second
That is about a 10% improvement. Either it is turboing, or something is more efficient in Ubuntu 15.04. Oh, I just remembered that it is now switched to 64-bit from 32-bit Ubuntu. Might be that, then.
This is untested by me but it has a number of nice traits:
jeppsson/Arduino-Communicator - Let your own Android application receive data from Arduino by listening to the "primavera.arduino.intent.action.DATA_RECEIVED" intent.
A python library that lets you chain together functions with dots into pipelines. Looks like jquery to me, is actually taken from Scala:
from functional import seq
seq([[1, 1], [2, 3], [5, -1]]).flat_map(lambda x: x).sum()
I wonder how it handles errors in the chain...
List of supported functions
List to List
- init: get everything except last element
- tail: get everything except first element
- inits: get subsequent inits
- tails: get subsequent tails
- drop: drop first n elements
- drop_while: drop first elements using f
- take: take first n elements
- take_while: take first elements using f
- map: map f onto sequence
- filter: filter sequence by f
- filter_not: filter sequence by not f
- reverse: reverse sequence
- distinct: return set of unique/distinct elements
- flatten
- flat_map
- group_by
- enumerate, zip_with_index
- partition
- slice
- zip
- sorted
List of (Key, Value) to List
- reduce_by_key
- group_by_key
List to Value
- head, first: get first element
- head_option: get first element or None
- last: get last element
- last_option: get last element or None
- reduce: reduce sequence using f
- fold_left
- fold_right
- count, len, size: get count of sequence
- any
- all, for_all
- max, max_by
- min, min_by
- find
- empty
- non_empty
- string: similar to mkString
- sum
Conversion to other types
- set
- list
- to_dict
This from the below linked page worked for me. Adjust /dev/ device to where your Arduino is. Mine is a Uno R3 clone.
To fix it, enter the command:
$ sudo usermod -a -G dialout
$ sudo chmod a+rw /dev/ttyACM0
My current rules for creating a web application in python
Summary: No exceptions, no classes, no frameworks.
Framework fatigue A framework is a mode, a global pragma on how to express the code in your application. In order to be efficient you need to understand how that framework works and it is unlikley that its modus operandi covers all the things that you want to do in a way that is flexible enough to match standard python. Instead use libraries, that gets hooked into your code, and only have a local impact on syntax and configuration.
Coming from the Zope/Plone world, it is very nice to throw all frameworky stuff to the wind and just express things in pure python. Now, frameworks do have their place, when you are solving something that many people have already solved for you, such as a content management system. But the moment your customer starts suggesting/dreaming up unique features you start fighting the framework. I chose to go with the bottle micro framework (which technically is a framework), that mostly stays out of the way and the module in my code that uses bottle is as thin as possible, and isolates the rest of the code from it.
Exceptions break the flow of the code, and it is harder to follow the execution of it. Furthermore in a web application you sometimes want to communicate several errors to the user, and not just the first one. Currently I use an extra return value in the form of an error object, that contains info if there is an error, some text that describes the error in a human readable way, and sometimes a recommendation of what url to redirect to.
class Status: def __init__(self): self.error = False self.redirect_address = None self.messages = [] def add_message(self, message): self.messages.append(message)
..and used in a function:
status = Status()
user_info = session.get_user_info_from_session(session_id)
if not user_info:
status.error = True
status.add_message("Please log in.")
status.redirect_address = '/login'
return (None, status)
In this way I can add error messages, and then return from a function when enough error messages have been added. That is, I can add errors that are not caused by each other, before returning. This works well with server side validation, where you can list all the things that need to be corrected back to the user, and not just the first one you test for (although obviously you do client side validation too, for usability reasons. But you still need to untaint the input).
Using an explicit return value for error or success is used in the Go language instead of exceptions. Idiomatic scala uses the entire result you get back with a return value. The latter would be highly magical in Python, so a separate status value is used.
Objects are in practice highly unpredictable in when it comes to how you're supposed to interact with them, and get data in and out of them. In practice, objects forces you to learn a mini configuration language for each class, expressed as methods, parameters and whatever state changes happens as a result of those method calls. With pure functions that complexity is confined to the parameters going in and the return values coming out. I have heard from Java programmers that the complexity of how to interact with objects is less of a problem if you have declared interfaces and an IDE that can tell you what's going on with inputs and outputs. However there is still way to much freedom to express yourself as a class author when it comes to how you go about performing a task.
Objects all the way down can make for something akin to a Rube Goldberg machine:
But it is also a question of how to do simple things such as setting a configuration value. Is that done by just setting an attribute, or calling a setter, or using a configuration object that somehow interacts with something?
Objects are often used to store state, which if it is mutable also messes up scaling up the application, and makes testing more difficult. In a web app pure functions also automatically makes your application scalable, since http is sessionless and each request needs to retrieve its state anyway from cookies, urls and server side stored data.
I actually did end up using some objects, as can be seen from the Status object in one of the code listings above.