This is Not a Brain Surgery

temperature measurement

Fri, 24 May 2013 05:30:12 GMT

As part of a small research project I have set to built a small sensor network to monitor temperatures at my home and office. Doing this I did some hardware (wireless Zigbee temperature sensors) and software to collect data from existing ones. The purpose of this post is to share quick links to scripts I have developed.

1. Main project page: https://github.com/vzaliva/xbee_temp_sensor contains Wireless sensor harware design and a collection scripts for:
a) Collecting data from my XBee-based sensors
b) Collecting data from 3M CT-50 Radio thermostat
c) Recording data from Weatherunderground.com weather stations
d) Submitting data to COSM (now renamed to XIVELY.com)

2. Data collection from a cheap USB temperature sensor and COSM submission: https://github.com/vzaliva/temper-python

3. Data collection from NEST thermostat and COSM submission: https://github.com/vzaliva/pynest

4. Data collection from Linux PC sensors (LMSENSORS) and COSM submission: https://github.com/vzaliva/lmsensors-cosm

If you want to learn more about the project, you can read my preliminary report or take a look at the presentation.

More on weather similarity

Thu, 28 Feb 2013 19:53:57 GMT

I probably already mentioned weather similarity service. Today I got curious: how the map of earth would look like if cities are organized by temperature similarity, no matter where they are located geographically. Below is a picture of what I got:

(I have labelled only cities with population over 2 million).

I wish I had a better way to show it in the browser. It would be nice to be able to zoom and show tooltips with city names. I have not find readily available JavaScript library for this and I do not have time to code this myself. If somebody interested in visualizing this I will be glad to share my data.

PPI fever

Thu, 22 Nov 2012 02:10:41 GMT

Once you switch to "retina" display on MacBook Pro, there is no going back. You just could not stop seeing huge pixels and artifacts on lower resolution screens. So, naturally when buying various devices with screens you want to keep an eye on PPI (pixels per inch). Here are some numbers (in decreasing order):

Nokia Lumia 920 - 332 PPI
iPhone 4/4S/5 Retina - 326 PPI
Samsung Galaxy S3 - 306 PPI
iPad 3 - 264 PPI
MacBook Pro 15-inch Retina display - 220 PPI
Amazon Kindle/Kindle Touch - 167 PPI
Microsoft Surface - 148 PPI
iPad 2 - 132 PPI
MacBook Air 13-inch - 128 PPI

(More numbers here)

Scala class students in percent of their country population

Thu, 22 Nov 2012 01:36:59 GMT

Few people at our company took Coursera "Functional Programming Principles in Scala" class. I think this is great when developers try to learn new languages and technologies and we actually pay a small bonus for taking such classes.

In one of his presentations Martin Odersky has shown a slide showing a distribution of students per country (also shown on this map). Unfortunately it is not very useful as it is not adjusted for country population. So my fiend Alex wrote a small Scala program to parse JSON and extract the data, and I have used Mathematica to adjust it by country population. The table with results, could be seen here. The bar chart showing top 40 countries in precent of their population taken the class looks like this:

USA is right behind Ukraine!

Amazon Instnat Video, Comcast and Google DNS

Wed, 07 Nov 2012 17:43:56 GMT

As Amazon Prime Customer I have access to Prime Instant Video. I have a PS3 with Amazon Instant Video application connected to my TV which I tried to use to watch some movies. Surprisingly the playback was freezing all the time and the application was complaining about insufficicent bandwidth. I have Comcast cable and SpeedTest clocks it at 30Mbps/5Mbps. Out of curiosity I tried instant video playback on my Mac and it has shown that it is getting only aroung 300Kbps of bandwidth.

To make long story short the problem was in my DNS settings. I was using Google Public DNS at home. Apparently this prevented Amazon CDN from properly detecting my location and streaming video from the closest server. Once I have switched back to Comcast DNS, video playback speed has improved and I am now able to stream movies in full HD.

I must report that after switching back to Comcast DNS I have noticed that my web page browsing become slower. Apparently Google DNS is indeed faster!

city maps image analysis

Fri, 19 Oct 2012 19:29:27 GMT

While reading recently about Armelle Caron work on city maps deconstruction it occured to me that city maps could be analyzed using mathematical tools for image analysis. I started by looking looking at city grids of Paris and New York:

Paris:

New York:

You can see lot of straight streets in NY while Paris is more curved. This could be more clearly seen by applying Hough Transform:

Paris:

New York:

(you can click on images for larger resolution versions)

We can also explore spatial frequencies of these maps using Discrete Fourier transform:

Paris:

New York:

I will not elaborate here how these images should be interpreted. If you are unfamiliar with them and interested to learn more you can read about these transformations which are well described in many other sources.

Horizontal Perspective Correction in Text Images

Tue, 12 Jun 2012 23:08:16 GMT

I was recently working on a computer vision problem which I would like to share with you. We were dealing with images of a restaurant menu containing mostly text. Images were taken with a mobile phone camera while the user held the menu in his hand. Typically the image captures just part of the page (page boundaries were not necessarily included). The challenge was to perform perspective correction on such images. We treated this as a practical, rather than an academic problem and thus did not aim to get the perfect solution, so we were ready to accept one offering only a partial improvement.

The main, hopefully new, contributions of our approach are:

Partial, horizontal-only perspective correction.
A new, statistical approach of choosing a horizontal vanishing point.

The problem of perspective correction is well studied. Please see references at the end of this post for related work. The basic idea is as follows: Given a pair of parallel lines, we can find a vanishing point. In a non-distorted image, all vanishing points lie at the line at infinity[1]. However, in a image projection, parallel lines may intersect at a real point. Given two pairs of lines which are supposed to be parallel (before projection), we can find two vanishing points which would define a line at infinity after perspective distortion. The idea is illustrated below. Vh and Vv are horizontal and vertical vanishing points respectively.

Now, working in homogeneous coordinates [5], we can build a homography [7], which would translate this line to an ideal line at infinity. Applying this homography to all pixels of a perspective-distorted image would allow us to reconstruct the original.

Our first task is to find two pairs of line in the distorted imags which were parallel in the original image. One obvious idea is to find horizontal lines corresponding to lines of printed text. The intersection of two such lines would give us a horizontal vanishing point. For the type of images we are dealing with, unfortunately there is a no easy way to detect vertical lines. In absence of vertical page margins in typical images, there are simply not enough vertical features with which to align such lines. So we must resign ourselves to partial perspective correction, using the horizontal vanishing point alone. This is done by assuming that the vertical vanishing point is located at infinity in the positive direction of the y axis. The coordinate of such a point in homogeneous coordinates would be (0,1,0), with the last zero indicating an ideal point (point at infinity).

Starting with a pre-processed image (already converted to black and white, we will apply Hough transform [6] to detect straight lines. Because we are interested in horizontal lines, assuming that we are not dealing with extreme cases of images being significantly distorted, we can limit the line angles that we consider to +-Pi/3 from the x axis. The transformation could be applied to a scaled down image, provided that the aspect ratio is preserved. This would help us to speed up the computations. Next, we will threshold the results of the Hough transform and convert the resulting detected lines from polar to homogeneous coordinates. As a result, we will have a small set of potentially horizontal lines. In theory any pair of them should suffice, but in practice, some detected lines may not correspond to horizontal lines of text, but represent a noise or other image features.

To select the two most suitable lines from this set, we will use a statistical approach. First we will build a pairwise intersection of all lines from the set. In homogeneous coordinates, the intersection of two lines is a cross product of their coordinate vectors. This gives us a set of a potential horizontal vanishing points. We will filter this set excluding points at infinity, points falling within the bounding rectangle of the projected image, and to exclude extreme cases of perspective distortion, the points located too close to the center of coordinates in the horizontal direction. The “too close” criteria is expressed as a threshold on an absolute value of the ratio of the x coordinate of a potential horizontal vanishing point to the image width. Each of the remaining points could be used to calculate a homography used to perform horizontal perspective correction:

where (a,b,c) is a projection of the line at infinity calculated as the cross product of Vx and Vy=(0,1,0). As a result of such correction, the two lines used to produce a chosen point will become parallel. Our working assumption is that because of the regular text line structure of the original image, most of the lines we detected were parallel. This allows us to define an evaluation metric of suitability for a homography as the standard deviation of the angles between all corrected lines and the x axis. The vanishing point producing the minimal standard deviation will correspond to the transformation which makes the set of lines the closest to parallel.

The resulting homography performs perspective correction, making our set of original horizontal lines to become closer to parallel after transformation. However, this does not make them actually horizontal. We need an additional affine rotation homography to achieve this. After perspective transformation, we will take the mean value of angles between lines in our set and the x axis as a rotation angle theta. The homography representing this rotation would be:

The final homography combining perspective correction and rotation is the combination of the two transformations

H = Ha Hp

which coincidentally have the form:

Sample results of the algorithm are shown below. Lines used for horizontal correction are shown in orange. The original image:

The corrected image:

References

[1] R. Hartley and A. Zisserman, "A multiple view geometry in computer vision", Second Edition, Cambridge University Press , 2004.

[2] P. Clark, “Estimating the orientation and recovery of text planes in a single image,” Proceedings of the 12th British Machine, 2001.

[3] V. Cantoni, L. Lombardi, and M. Porta, “Vanishing point detection: Representation analysis and new approaches,” Image Analysis and, no. Iciap, 2001.

[4] L. Jagannathan, “Perspective correction methods for camera based document analysis,” on Camera-based Document Analysis and, pp. 148-154, 2005.

[5] Wikipedia contributors. Homogeneous coordinates. Wikipedia, The Free Encyclopedia. May 22, 2012, 07:12 UTC. Available at: http://en.wikipedia.org/w/index.php?title=Homogeneous_coordinates&oldid=493787558. Accessed June 12, 2012.

[6] Wikipedia contributors. Hough transform. Wikipedia, The Free Encyclopedia. June 11, 2012, 00:12 UTC. Available at: http://en.wikipedia.org/w/index.php?title=Hough_transform&oldid=496983042. Accessed June 12, 2012.

[7] Wikipedia contributors. Homography. Wikipedia, The Free Encyclopedia. March 12, 2012, 06:33 UTC. Available at: http://en.wikipedia.org/w/index.php?title=Homography&oldid=481473030. Accessed June 12, 2012.

time based parental control for web sites

Sat, 25 Feb 2012 02:18:51 GMT

My 9 y.o kid is using Internet. I see nothing wrong with that. Internet is no longer an optional part of this world and he needs to be friendly with it. He reads Wikipedia, visits LEGO-related sites, etc. I have enabled parental controls on his Mac, so hopefully some inappropriate web sites are blocked. The only problem I have is amount of time he spends on YouTube. YouTube could be an amazing educational resource but at the same time it could be an amazing time waster. I decided to limit amount of time he spends on YouTube, like many parents limit amount of time their kids spend in front on TV. I found no easy way to do this short of blocking his complete access to computer according to pre-defined schedule. But I wanted to block only YouTube and let him use everything else. Below I will share my solution to the problem. It is a bit involved, and requires you to have Linux box on your home network, but this is the only working solution I found so far.

The trick is to use HTTP proxy (Squid) with squidguard plugin. On RHEL you can install and activate required packages using following commands:

yum install squid squidguard squidguard-blacklists
chkconfig squid on

Let us start with default config file:

cd /etc/squid
cp squidguard-blacklists.conf squidguard.conf

Now edit squidguard.conf, adding following lines at the bottom:

time playhours {
weekly * 19:00-20:00
}

src all {
ip 10.0.81.0/24
}

### ACL definition
acl {
all within playhours {
pass good !bad !adult !aggressive !hacking !warez any
redirect 302:http://localhost/access-denied.html?url=%u
} else {
pass good !bad !adult !aggressive !audio-video !hacking !warez any
redirect 302:http://localhost/access-denied.html?url=%u
}

default {
pass none
redirect http://localhost/block.html
}
}

My home network is 10.0.81.0/24. This config allows to access video web sites (which include YouTube) only
during 1 hour: from 7pm to 8pm every day. You can devise more complex schedule if you want to. In addition
to blocking YouTube this config blocks various adult and hacking web sites at all times, as extra level of protection.

To test your config you can use this handy command:

echo "http://www.youtube.com 10.0.81.1/ - - GET" | sudo -u squid squidGuard -c /etc/squid/squidguard.conf -d

To activate it, edit squid.conf adding the following line:

url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf

Now all you need to set up your child computer to use HTTP proxy, pointing to IP of squid computer and default
port 3128. Unfortunately due to the bug in MacOS versions up to Lion you need to disable parental controls on the account to make proxy settings work.

As an extra measure of protection, you can use OpenDNS free parental controls. Here is how my OpenDNS dashboard looks like:

After setting up OpenDNS account all you need to do is to add the following line to squid.conf:

dns_nameservers 208.67.220.220 208.67.222.222

Hint: if you have dynamic IP, you can use ddclient to update it at OpenDNS site using this config.

And here is what is shown if YouTube is accessed during not permitted time slot:

Mobile user interface

Sun, 21 Aug 2011 21:27:59 GMT

Designing mobile user interface is not very complex. Just use common sense, think about user, and keep an eye on limited screen estate. But, some people just do not get it. For example, Foursquare Android App, which I am using daily, is driving me nuts. Take a look at the first screen I see when it is started:

It opens in "Friends" tab, which supposed to show me where my friends are. OK. Makes sense. What do I see:

1. It shows me where I am. Doh! I know where I am. No need to remind me. I want to see my friends, and given limited screen space it does not make sense to waste it on reminding me where I am now.

2. It shows me same information again, this time under "Last 3 hours" section. This is just plain stupid.

At this point, 1/3 of screen space which could have been used to show my friend’s locations have been wasted just to remind me that I am Starbucks where I have checked in just 2 minutes ago. Taking into account additional space used by three levels of toolbars on top, just enough useful screen space left to shows me the locations for only two of my friends.

growing money

Tue, 21 Jun 2011 23:03:26 GMT

There is a lot of buzz about BitCoin in the news recently. Most people are exited about distributed, untraceable, and unregulated nature of this new currency. However, there is another overlooked aspect of it, related to the practice of "bitcoin mining." Anybody now can buy some hardware and start minting his own bitcoins. Initial investment in hardware aside, all you need now is to spend some electricity to produce money (bitcoins). Depending on your electricity costs and bitcoin exchange rates you can make small but steady monthly income.

An interesting aspect of this electricity-to-money conversion is that electricity does not need to be transferred far and could be consumed very close to a generator producing it. For example, you can have a module with solar battery on top and bitcoin-generating computer inside which you can put anywhere to get free money literally growing under a sun!

For example, the picture below shows 1000w Solar Photovoltic System. It generates enough electricity to power a PC with 2-4 powerful GPU cards, capable of producing very decent bitcoin mining performance.

Stolen laptop registry

Mon, 28 Feb 2011 02:50:59 GMT

My daughter is a UCSC student. Recently, her MacBook was stolen from her dorm room. She has filed a burglary report with campus police, but recovery chances are slim. Not to slander character of UC students, but I think there is a good chance the laptop will resurface somewhere on campus someday. Once it connects to the campus network it could be identified by MAC address. From there, hopefully, the identity of thief could be tracked down. I was able to find MAC address of stolen laptop, and my daughter gave it to campus IT. However, I am not sure if they would take the task of tracking stolen laptops seriously.

This made me think that there might be a crowd sourced solution to this problem. Imagine a site, where one can report stolen computer along with its MAC address. Now, the volunteers would run a small daemon on their laptops. The daemon will be constantly monitoring the ARP table and check if any of nearby MAC addresses are in the list belonging to stolen machines. If match is detected a report along with spotting time and the location is sent to the owner. If enough people participate miscreants could be spotted on campuses, in hotels, at offices, cafes. This should work for all WiFi or ethernet equipped devices: computers, laptops, tables, mobile phones, game consoles, etc.

Obviously, there are some privacy issues. Sending somewhere MAC addresses of nearby computers could be viewed by some people as a privacy violation. This could be addressed by using one-way hashes and sending them instead of MAC addresses. Second concern is what if somebody puts into the system a MAC address of somebody else laptop with purpose to track this person. This issue is more difficult to address. Maybe the site should require submitters to provide their real identity (verified by credit card of by other reliable means) or accept new records only if police report has been filed and a copy has been enclosed.

godaddy backup with rsync over ssh

Sat, 05 Feb 2011 18:31:55 GMT

I have recently moved one of my web servers containing mostly static pages to GoDaddy hosting. Before it was hosted at Linux machine and I was backing it up using weekly cron sript using rsync. With new hosting I had to set up a new backup procedure. The problem us that although GoDaddy gives you SSH access they do not provide rsync on their machines. I was toying with idea of cross-compiling it elsewhere and bringing a binary with me, but I found a simpler solution using sshfs. Here are step by step instruciton:

Definitions:

www.yourhost.com - your go daddy web host (assuming it has ssh access)
username - your login name for ssh access to go daddy host
backuphost- a Linux machine with root access, at which you will be backing up your web site.

on backup host:

1. As root, generate new ssh key without passphrase using ssh-keygen command. Call it backup.dsa. (normally stored in /root/.ssh/backup.dsa)

2. Add the following lines to your /root/.ssh/config

Host www.yourhost.com
User username
IdentityFile ~/.ssh/backup.dsa

3. Make sure you have empty mount point (mkdir /mnt)

on www.yourhost.com:

1. Add contant of the file backup.dsa.pub (one you generated on backup host) to ~/.ssh/authorized_keys

Now we are all set. Let us try it:

First just try to see if your ssh works. As root issue the command:

ssh www.yourhost.com

It should let you in without password. If this does not work add -v flag to ssh command and proceed with regular troubleshooting of ssh public key authentication. Most likely you will need to fix permissions on ~/.ssh directory and its contents on the remote host.

Once this worked, you can mount remote filesytem:

sshfs www.yourhost.com: /mnt

And back it up:

rsync -az -qq /mnt/html/ ~/backups/www.yourhost.com

Do not forget to unmount when you done:

umount /mnt

This is a quick and dirty solution. Most likely fusefs options could be tweaked to get even better performance. You can add these commands to script file in /etc/cron.weekly/ on backup host to implement automatic weekly backups.

Android keyboard

Tue, 25 Jan 2011 23:59:27 GMT

Switching between Nexus One and iPhone I could not help noticing that my typing error rate is significantly lower with iPhone. Both devices have about the same screen size and similar keyboard layout. That made me wondering what makes iPhone keyboard superior. Is it the layout, for example spacing between the keys? Smarter algorithms? With help from Quora I have been able to disover that one of the key differences is an algorithm used by iPhone to adjust target area based on next key prediction:

"Although the size of the keys displayed on an iPhone/iPod Touch keyboard does not change during typing, the iPhone's predictive algorithm will subtly increase the "target area" of what it thinks your next keystroke will be. This helps reduce mistypes."

Alternative Android keyboard called TickButtons using similar idea, except it does change visual size of the keys:

Luckly there are many alternative keyboards available for Android. I was looking for a conventional keyboard, so I have exluded ones like Swype. After trying several of them I was able to find one which allows me to type much more accurately on Android. Not only it supports target area resizing as iPhone does, it also have a calibration mode where you train it to the way you are pressing the keys. I am now happy user of Smart Keyboard.

smarter car navigation

Mon, 17 Jan 2011 01:16:34 GMT

Here is the situation: you are driving a car, perhaps using your GPS device for navigation and want to stop for a quick bite or a cup of coffee. You would probably first think about using list of nearby restaurants offered by your GPS device, or perhaps an application like Yelp to see what are the good places to eat nearby. The problem is that location search offered by most applications is not particularly optimized for such simple and common situation. The gist of the problem is that it is assuming that you are static and does not use information about your route and roads into account.

For example it would show you all restaurants in, say, 5 mile vicinity from whenever you are. But most likely you will not be interested in the restaurants which are located behind you. You would rather not spend time turning around and going back. So in simple case your search area should be shaped not like a circle with radius of 5 miles with you in the center, but rather as a rectangle extending along your route with some reasonable distance on both sides. This would make your search aware of your route.

In more general case I wish my navigation system to be clever enough to search in a “detour” mode, assuming than whatever I am searching for is just a stop on my path and I would want to continue to my ultimate destination afterwards. In this case it could use different distance metric to filter and order search results. Now it orders them just by distance from my location. What I really care is how much detour to given place will extend my route. For example, let us call my current position A and my final destination Z. There are two potential restaurants, say, B and C which fit my search criteria (Indian food most likely, in my case). Instead of using distance AB and AC it should build a route A-B-Z and A-C-Z and compare overall time from A to Z over each of these routes. It would be especially nice if it would use traffic information, if available.

OWI Robotic Arm Edge USB protocol (and sampe code)

Fri, 05 Nov 2010 21:15:16 GMT

In my last post I have described OWI-007 Robotic Arm parallel port protocol. They have a newer, more advanced model called Robotic Arm Edge:

You can use USB interface module to connect it to PC. Unfortunately official software works only under Windows and do not provide API. I have been able to reverse engineer their USB protocol and write sample code which allows to control the arm from Mac and Linux.

This is USB device with vendor id 0x1267 and product id 0. The device is controlled by 3 byte commands sent via USB control transfers. Command format is byte0, byte1, byte2. Each byte controls a group of arm features. All motors could be controlled independently. Most commands start action which is continued until next action is signalled. Byte '00' universally used as stop action.

Bit numbering in byte:

7 (most significant)

0 (least significant)

Meaning of bits in each byte:

Byte 0

First byte controls grip, shoulder and wrist. Bits are assigned as following:

Bit numbers	Controls	Bit combinations
0,1	Grip	00-do not move, 01-close, 10-open
2,3	Wrist	00-do not move, 01-move up, 10-move down
4,5	Elbow	00-do not move, 01-move up, 10-move down
6,7	Shoulder	00-do not move, 01-move up, 10-move down

Sample commands, activating single motor:

Byte 0	Command
00	stop grip, wrist, elbow and shoulder movement
01	grip close
02	grip open
04	wrist up
08	wrist down
10	elbow up
20	elbow down
40	shoulder up
80	shoulder down

Sample commands activating 2 motors simultaneously:

Byte 0	Command
11	grip close and elbow up
82	grip open and shoulder down

Byte1

Second byte controls base.
Bits are assigned as following:

Bit numbers	Controls	Bit combinations
0,1	Base	00-do not move, 01-rotate clockwise, 10-rotate counter-clocwise

Sample commands:

Byte 1	Command
00	stop base rotation
01	rotate base clockwise
02	rotate base counter-clocwise

Byte 2

Third byte controls LED light inside the grip.
Bits are assigned as following:

Bit numbers	Controls	Bit combinations
0	LED	0-off, 1-on

Sample commands:

Byte 2	Command
00	LED off
01	LED on

Sample source code using libusb:

#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <string.h>

#include <libusb-1.0/libusb.h>

#define EP_INTR	(1 | LIBUSB_ENDPOINT_IN)

#define ARM_VENDOR       0x1267
#define ARM_PRODUCT      0
#define CMD_DATALEN      3

libusb_device * find_arm(libusb_device **devs)
{
	libusb_device *dev;
	int i = 0;

	while ((dev = devs[i++]) != NULL) {
		struct libusb_device_descriptor desc;
		int r = libusb_get_device_descriptor(dev, &desc);
		if (r < 0) {
			fprintf(stderr, "failed to get device descriptor");
			return NULL;
		}
		if(desc.idVendor == ARM_VENDOR &&
		   desc.idProduct == ARM_PRODUCT)
		  {
		    return dev;
		  }
	}
	return NULL;
}

int main(int ac, char **av)
{
    if(ac!=4)
    {
        fprintf(stderr,"Usage: armedgetest CMD0 CMD1 CMD2\n");
        return 1;
    }

    unsigned char cmd[3];

    cmd[0]=(unsigned char)strtol(av[1],NULL,16);
    cmd[1]=(unsigned char)strtol(av[2],NULL,16);
    cmd[2]=(unsigned char)strtol(av[3],NULL,16);
    
	libusb_device **devs;
    libusb_device *dev;
    struct libusb_device_handle *devh = NULL;
	int r;
	ssize_t cnt;

	r = libusb_init(NULL);
	if (r < 0)
    {
	    fprintf(stderr, "failed to initialize libusb\n");
	    return r;
    }

    libusb_set_debug(NULL,2);
    
	cnt = libusb_get_device_list(NULL, &devs);
	if (cnt < 0)
		return (int) cnt;
    dev=find_arm(devs);
    if(!dev)
    {
	    fprintf(stderr, "Robot Arm not found\n");
	    return -1;
    }

    r = libusb_open(dev,&devh);
    if(r!=0)
    {
	    fprintf(stderr, "Error opening device\n");
       	    libusb_free_device_list(devs, 1);
             libusb_exit(NULL);
	    return -1;
    }


    fprintf(stderr, "Sending %02X %02X %02X\n",
            (int)cmd[0],
            (int)cmd[1],
            (int)cmd[2]
    );

    int actual_length=-1;
    
    r = libusb_control_transfer(devh,
                                0x40, //uint8_t 	bmRequestType,
                                6, //uint8_t 	bRequest,
                                0x100, //uint16_t 	wValue,
                                0,//uint16_t 	wIndex,
                                cmd,
                                CMD_DATALEN,
                                0	 
    );
    
    if(!(r == 0 && actual_length >= CMD_DATALEN))
    {
        fprintf(stderr, "Write err %d. len=%d\n",r,actual_length);
    }
    
    libusb_close(devh);
	libusb_free_device_list(devs, 1);
	libusb_exit(NULL);

    fprintf(stderr, "Done\n");
	return 0;
}

Sample usage:

./armedgetest 00 00 01
sleep 3
./armedgetest 00 00 00

will turn the light on for 3 seconds and then would turn it off. It have been tested on MacOS X, but should work on Linux as well.

OWI-007 Robotic Arm parallel port protocol

Fri, 29 Oct 2010 21:03:24 GMT

One of my projects involes controlling a robot arm. We were using Robotic Arm Trainer OWI-007 from OWI. It is cheap (under $100). We were using their parallel port interface. OWI does not offer official API, but it was easy to figure out command sequences. Here are bytes one need to send to the parallel port, to control the arm:

stop_arm = 0x05,
grip_open = 0x0D,
grip_close = 0x00,
elbow_up = 0x15,
elbow_down = 0x01,
wrist_cw = 0x04,
wrist_ccw = 0x81,
shoulder_up = 0x03,
shoulder_down = 0x45,
base_ccw = 0x02,
base_cw = 0x26

Sending one of these commands will make an arm perform specified motion until STOP (0x05) or other command is sent. Only one motor at a time could be engaged.

There is no speed control, but we have been able to implement a limited speed control using Pulse With Modulation approach.

Now we are looking at their newer model, Robotic Arm Edge which only comes with USB interface. OWI "don't offer any support for 3rd party programming for the USB interface", but I hope we can figure out their USB protocol on our own.

Google Apps

Wed, 06 Oct 2010 21:15:22 GMT

My company has recently switched to Google Apps (paid edition). It is not cheap: $50 per user annually, but we decided it is worth it, comparing to cost of supporting our own email infrastructure. Google was kind enough to assign as a support engineer to help us with migration. We did migration on our own, and just needed help with few small issues. Before boring you with details let me go straight to conclusions, in case you do not want to read the rest:

1. Core functionality of Google Apps works pretty well - we are happy
2. Do not expect much support. My impression is that Google is doing whatever they think they should be doing, and do not care about you or your money. Just be grateful to be able to use their stuff and shut up. [This is similar to Apple's attitude, except Apple cares about your money]
3. Knowing all this, would I still switch to Google Apps? Emphatically YES.

Sample issues:

Problem: There is no way to browse global address book, although you can search in it.
Suggested Solution:
1. Export CVS and email to all your employees. Ask them to import to their private address books.
2. Use API (they actually offered to send me a link to API docs!) [I guess the assumption is that I should write my own interface to their address book]

Problem: There is no way to change owner of documents on Google Docs from @gmail.com address to google apps user (it complains about different domains)
Suggested Solution: Export all documents in Ms Word/Excel formats and re-import them, losing history, sharing settings, etc.

Problem: Some problems with IMAP mailbox migration (Feature of apps control panel)
Suggested Solution:
1. Although it is offered via Control panel it is deprecated and should not be used. It is not supported.
2. Although I am not using Microsoft Exchange, I should install "Google Apps Migration for Microsoft Outlook". It will import IMAP as well. It requires Outlook to be installed. I will need Windows machine for this. Then it will proceed to download 4Gb of my mailboxes via IMAP and will re-upload them to Google Apps.

Problem: There was typo in name of one of users which we fixed. In the list of accounts it shows correctly, but in auto-completion in email and elsewhere it still shows mistyped one, one week after it was fixed.
Suggested Solution: Usually changes to user real names take 24 hours propagate. [When I pressed on saying that a week have passed, they said that this is known issue which will be fixed, eventually. No ETA, no workaround)

Problem: You can create calendar, and in sharing options make it shared with all people in the company. But co-workers have no way to find it - search does not show it.
Suggested Solution: You need to get cryptic calendar ID from sharing settings (it looks like email address) and email it to all your employees, and as them to add to their calendars using this ID.

.DS_Store and SVN

Fri, 23 Jul 2010 18:37:53 GMT

It is kind of obvous tip, but I want to share it anyway:

I was annoyed by .DS_Store files left by Finder in my source directories. SVN reports them as unchecked. An easy way to fix it is to add .DS_Store to list of global igore patters. See global-ignores in your ~/.svn/config. While at it, consider also adding '.pyc' there is you are a python programmer ;)

safari vs. chrome - user perspective

Mon, 14 Jun 2010 20:11:35 GMT

I've been oscillating between Safari and Google Chrome last few weeks without being able to decide which one I like better. Current status - I am back to Safari. Few reasons:

1. PDF support - I often read PDF documents from the Web and Chrome method of using Google as PDF viewer is not good enough for me. I want real PDF viewer (Adobe or Preview) inside my browser window.

2. Dictionary support: I like to be able to use Apple build-in dictionary/spelling tools in my browser. It does not work in Chrome.

3. Email copy-paste: I am using internal Wiki for research and project notes. If I select some part of wiki page, copy and paste into email - in Safari I get formatting and embedded images. With Chrome I only get text.

4. 1Password plugin - Chrome version lacks many features

5. Safari 5 "Reader" mode - looks like something I could use.

6. Ability to configure handler for RSS URLs, so I can subscribe to RSS feeds in NetNewsWire from web page without copying and pasting URLs.

In terms of speed and content rendering both browsers are looking pretty good, but small things like ones listed above are important for me.

High-performance cloud servers

Thu, 15 Apr 2010 17:37:20 GMT

For a project I am working on I need to use a lot of CPU power (training Neural Networks). Instead of buying fast server I decide to evaluate cloud-based solutions. For a long time Amazon Web Services user like myself EC2 was the first choice. The fastest machine they offer is:

Amazon High-CPU Extra Large Instance
7 GB of memory
20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each)
1690 GB of instance storage
64-bit platform

The estimated price for non-stop usage during 30 days is $489.60.

Rackspace cloud do not offer high-CPU servers.

GoGrid offers "6 x Xeon" as the fastest sever. The details vere scarce and I have contacted their support for clarification. They quoted me about $500 per month and gave $100 credit to try their service. I did. As you create an instance you could not implicitly choose CPU type. You can chose memory size and more memory you chose the faster CPU you will get. I had to call their support to find this out. So to get their fastest CPU you need to order 8Gb RAM machine.

Then I have fired up two machines, one on EC2 and one on GoGrid and run a small (admitedly unscientific) test:

EC2:
real 717m21.988s
user 724m56.935s
sys 0m45.715s

GoGrid:
real 826m18.441s
user 809m10.078s
sys 1m7.850s

So as we can see EC2 is roughly 1.1 times faster. Then I looked at their panel and discovered that they misquoted me their price. I checked again their web site. To run 8Gb instance for 30 days will cost me $1094! Which is twice as much than EC2!

So their unethical bait&switch sales strategy aside, GoGrid is slightly slower and much more expensive than EC2.

iPad screen size

Sat, 10 Apr 2010 18:53:18 GMT

To format some content for iPad screen, I was looking for exact screen dimensions. In official specs it states:

9.7 inch diagonal
1024-by-768-pixel resolution at 132 pixels per inch (ppi)

From that we can deduct some useful numbers, such as:

4:3 aspect ratio
7.76 by 5.82 inch
197.1 by 147.8 mm
5.197 pixels/mm

unscientific google chrome performance test

Mon, 28 Dec 2009 00:37:54 GMT

Google is advertising Chrome as a very fast alternative to other browsers. I am also trying to use Google Wave and I have noticed that on long waves, scrolling in Safari becomes a bit slow. So I decided to try Chrome. Surprisingly scrolling long wave on 1050x1680 (portrait mode) display on 2.8GHz MacBook Pro is much smoother and faster under Safari than under Chrome. In fact under Chrome scrolling is so jerky it is almost unusable.

google app engine name cybersquatting

Fri, 18 Dec 2009 00:26:06 GMT

While trying to come up with a name for a new application which will be hosted at google app engine I have realized that whatever name I can come up with is already taken. I suspect somebody wrote a bot to register all dicitionary words and their combinations. What would be any other explanation for names like "orangepotato", "fluffyrazor" and "suicidesquirrel" being already taken.

yubico and a false sense of security

Wed, 05 Aug 2009 22:42:07 GMT

I was about to buy a Yubico key today. It is a security token which generates on time passwords (OTP) which could be used in conjuction with good old password to implement two-factor authentication. There are many things I like about their product. The form factor is great. They use a very neat trick of simulating USB keyboard, which makes it very easy to integrate with OS without a need for a special drivers. The key already have been integrated with many applications as described on their Wiki. They are also support open-source and try to be as open about their platform as possible.

Before purchasing my yubico key I took my time too read their manual to understand better how their security works. The basic idea is very simple: each time you plug the key in and press a button it generates one time password. Yubico OTP is specially encoded AES encrypted block of data. The data consists of token id, random number, incrementing counter and some other fields. Let us take a look how it holds against some attacks:

1. Replay attack. Because the encrypted block includes incrementing counter, it seems to be safe from this kind of attack. Imagine we have sent OTP with counter=1. Malicious hacker, listening on the line have recorded it and tries to reuse it to log in again. However the server remembers last counter value and will not accept second authentication attempt unless counter value is greater than last used value for the token. So far so good. But here is the quote from their manual: "The non-volatile counter is compared with the previously received value. If lower than or equal to the stored value, the received OTP is rejected as a replay." Quite sensibly they do not require next counter value to be exactly previous one plus one - any accidental button press when focus is in the wrong input field will break key/server synchronization. So if you try to use your key with two different servers, OTP generated for service A could be recorded and re-used for server B until user logged in to server B:

1. User logs in to Service A with OTP-A1
2. Hacker records OTP-A1
3. Hacker logs in to Service B with OTP-A1
4. User logs in to Service B with OTP-B1

2. Man-in-the-middle-attack - Imagine a piece of malicious software, pehaps in a form of a USB driver, which sits between the key and the server. User plugs in his key and presses a button for a first time. The driver records the OTP1 but either not sends it to the server, or sends a slightly modified (incorrect) version. The authentication fails. At this point user will likely just try again, this time emitting OTP2. The software will however will use OTP1 and will remember OTP2 for future use. At this point, at each login, the previous token will be used. The last token will be saved and can be used anytime later to gain unauthorized access on user behalf.

3. Server-side compromise. Imagine the server with which you use with your Yubico key is compromised. The hackers will gain access to AES key and token ID. This is all information they need now to simulate your token to generate one time passwords and try to log in to other systems set up to use the same token.

In conclusion, first of all I must say that using Yubico in addition to password or some other authentication method is certainly an improvement in your security. But while doing so you should be aware that addition security provided by the key is not iron-clad and vulnerable to attacks listed above. The risks are slightly lower if you are not using same token with multiple services.

contemplative debugging

Wed, 08 Apr 2009 18:28:01 GMT

After reading this post, I want to share with you my take on the subject:

More than a decade ago I was using SGI Indigo II as my primary development workstation. We were writing in C++ and I have quickly discovered that SGI C++ compiler (cfront based) is pretty bad. To give you idea how bad it was - you can actually compile code which invokes pure virtual function, which, of course, was causing a crash during execution. So, we decided to switch to GCC, which was also not very mature back then in C++ department but was certainly a step ahead of SGI C++. The problem with G++ on SGI platform at that time was that for some reason it could not generate debug information in Irix executables. I think it has something to do with COFF executable format used by Irix 4.x, or something like that. So I had to learn to live without debugger. I was relatively immature programmer without set work habits and I did not yet have preference to particular debugging technique. I worked like this for about a year, and then I switched to some other Unix platform after this, the "damage" have been done: I never had a need to use debugger ever since.

This custom helped me several times during my career:

1. Debugging PROLOG code where step-through debugging techniques are not applicable
2. Programming mobile devices. (I remember we used to change a colour of a pixel in a upper right screen corner or emitting series of beeps as debugging techniques)
3. Debugging Haskell programs (lazy evaluation, etc.)
4. Debugging a distributed systems
5. Diagnosing a problem reported by a customer, when you do not have access to actual device

My friends and I were joking that we have invented a new debugging technique which we called "contemplative debugging" ("созерцательная отладка" in Russian). We used to practice a pair programming. It was before "extreme programming" become poplular, and was mostly because we just do not have enough workstations. So, once the bug was reported, my partner and I intensively stared into suspect code until one of us emits cry of joy, discovering the problem. This technique, in many cases, turned out to be quite efficient, compared to firing up debugger and stepping through the code to get to the problem.

P.S. I am not religious about not using debugger. I use it occasionally if need arise (which is not very often).