Posts Tagged programming

Installing Play Framework on OpenBSD 4.6

OpenBSD
OpenBSD is a free, reliable and secure operating system. From a configuration standpoint it is both minimal and simple, which is great for those who want to get started quickly. For the purposes of this tutorial, it is assumed the user has already installed OpenBSD. If not, check out openbsd101.com for guides on installation etc.

Play Framework
The Play Framework is a java based web programming system, that includes the enterprise features of java with the methodology of Ruby on Rails or Django. You can view an introductory screencast at their website which shows just how easy and powerful it is.

Allow your user to sudo
Since this blog is aggregated on many Ubuntu sites, we will use the sudo facility to run commands instead of root. To enable sudo the same way Ubuntu does:

  1. su – # Get root
  2. visudo
  3. Uncomment the line “%wheel ALL=(ALL) SETENV: ALL”

Install packages
For the play framework the launch scripts are in python. Zsh is installed for a better shell, and vim is installed for a full featured editor.

export PKG_PATH=ftp://ftp.openbsd.org/pub/OpenBSD/4.6/packages/i386/
sudo pkg_add zsh jre-1.7.0.00b59p0 wget python-2.6.2p0 vim-7.2.190p1-no_x11 unzip
sudo ln -sf /usr/local/bin/python2.6 /usr/bin/python
sudo ln -sf /usr/local/bin/python2.6-config /usr/bin/python-config
sudo ln -sf /usr/local/bin/pydoc2.6  /usr/bin/pydoc

Install Play Framework

cd /usr/local
sudo wget http://download.playframework.org/releases/play-1.0.zip
sudo unzip play-1.0.zip

Start your project

cd /var
sudo /usr/local/play-1.0/play new ourappname
sudo chown -R ourusername ourappname
cd ourappname
# Set java home -- you can set this permanently in /etc/login.conf or in a startup script
export JAVA_HOME="/usr/local/jre-1.7.0/"
/usr/local/play-1.0/play run

Done
Your test app is now listening on port 9000 of the systems IP. That is all there is to it. Make sure to check out the excellent documentation available for the Play Framework.

play

Tags: , ,

Python and real time graphical analysis

I have a camera which has a motor attached which I can rotate using a serial cable. I figured it would be fun to have this camera analyze the webcam shots and turn in any direction there was motion. I pulled out python and pygame, and created a prototype. Unfortunately, I can’t make python go very fast. I made two test cases, 1 in C and 1 in python, to figure out if it would be worthwhile to rewrite it:

array-speed-test.c

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <sys/time.h>

void compare_arrays(int **screen1, int **screen2)
{
    int x;
    int y;
    int diff;
    int mult;

    for(x = 0; x < 1920; x++) {
        for(y = 0; y < 1080; y++)
        {
            mult = screen1[x][y] * screen2[x][y];
            diff = abs(screen1[x][y] - screen2[x][y]);
        }
    }
}

int main(int argc, char **argv)
{
    srand(time(NULL));
    printf("Generating arrays...\n");
    int **screen1;
    int **screen2;
    int i = 0;
    int j = 0;
    struct timeval now;
    struct timeval end;
    int usecs_passed;

    screen1 = malloc(1920 * sizeof(int *));
    screen2 = malloc(1920 * sizeof(int *));
    for(i = 0; i < 1920; i++)
    {
        screen1[i] = malloc(1080 * sizeof(int));
        screen2[i] = malloc(1080 * sizeof(int));
    }

    for(i = 0; i < 1920; i++)
    {
        for(j = 0; j < 1080; j++)
        {
            screen1[i][j] = rand() % 255;
            screen2[i][j] = rand() % 255;
        }
    }

    printf("Comparing arrays...\n");

    gettimeofday(&now, NULL);
    compare_arrays(screen1, screen2);
    gettimeofday(&end, NULL);

    usecs_passed = end.tv_usec - now.tv_usec;

    printf("Time passed: %dms\n", (usecs_passed / 1000));
    for(i = 0; i < 1920; i++)
    {
        free(screen1[i]);
        free(screen2[i]);
    }

    return 0;
}

And my python code:

array-speed-test.py

#!/usr/bin/python
import random
import time
from math import fabs

def generateArray():
    array_to_gen = [None] * 1920
    for i in range(0, 1920):
        array_to_gen[i] = [None] * 1080

    for x in range(0,1920):
        for y in range(0, 1080):
            array_to_gen[x][y] = random.randrange(0,255)

    return array_to_gen

def compareArrays(screen1, screen2):
    for x in range(0, 1920):
        for y in range(0, 1080):
            diff = fabs(screen1[x][y] - screen2[x][y])
            combo = screen1[x][y] * screen2[x][y]

if __name__ == "__main__":
    print "Generating arrays..."
    screen1 = generateArray()
    screen2 = generateArray()

    print "Created two screens.  Comparing..."
    startTime = time.time()
    compareArrays(screen1, screen2)
    print "Time taken: " + str((time.time() - startTime) * 1000) + "ms"

So far, the C program runs in 25ms, while the python program consistently takes 1100ms. Might have to ditch python for real time analysis, unless someone wants to point out how I am doing this completely wrong (I am assuming the comments will be use Numpy?)

Tags: , ,

Boring Friday – What are you doing that is fun?

Suffering from a little burnout after delivering a few projects. For those coders out there, what are you working on thats fun / exciting?

I will start:

Favicon_normalFollowing up from my previous post, the most fun thing I did was compete in the Engine Yard Contest.. I created a python client, which communicated the results back to my webserver every 23 minutes via xml. On the webserver itself, I used the watch command to provide stats that auto update: ‘watch -n 30 “cat /tmp/results | sort | uniq | head”‘. The results had their score first, with lowest being the best like golf.

My favorite webpage I have viewed today has been the Arch Linux July Screenshots thread: http://bbs.archlinux.org/viewtopic.php?id=75154

Tags: ,

Annoying people with code: A gentle introduction to C# and Mono Part 3: Creating a GUI (Graphical) Mono / C# Program

In this post we will cover how to make a basic GUI application using Mono along with Monodevelop. If you haven’t already, you will want to read part 1 and part 2 of this series before this article.

Monodevelop
Monodevelop is the tool we are using as our editor. This is Linux’s equivalent of Visual Studio. It can be used to code, debug and design your application, and can link in with revision control systems. This article was written using version 2.0, if your version is not 2.0 you may see slight differences.

Create your project
After starting Monodevelop, go to File -> New -> Solution. You will be presented with a prompt asking which kind of solution we are starting. Click on C# -> Gtk# 2.0 Project as shown below:

new-solution

Read the rest of this entry »

Tags: , , ,

Annoying people with code: A gentle introduction to C# and Mono Part 2 – Data Types

In part one of this series we covered making a basic “Hello World” plugin in Mono / C#. Today we are going to dive into different types of data, and how to define them. Show me part 1! Also if you thought part 1 was ugly, I added a new javascript syntax highlighter so I recommend you read it on my blog instead of planet.ubuntu.com.

Variables
A variable is something that stores a value. In different programming languages, these are handled differently. C# is called a static / strongly typed language as you have to tell it what variables are. To explain, I will use python in comparison:

Python Example

x = "This is a string"
y = 5

As you can see, we didn’t have to tell Python anything about what type of variables they are. In C# / Mono, we would do this like this:

C# Example

string x = "This is a string";
int y = 5;

I will tell you, after my years of development, that I program a ton of stuff in PHP / Python today. One of the worst and best parts of these languages is that they are dynamically typed. The fact that C# requires variables to be declared with a type before using makes life easier for testing and reading. There is a stack overflow post on what static typing gives you: Read the stack overflow post.

Variable Naming
Before we continue, you will notice in the top example I used the variable names x and y (to illustrate that the variable names really mean nothing to the compiler / interpreter. If I named them myString you might have been confused). If you do this in real life, I will find you and give you a thorough verbal lashing. I generally use camel case to name variables. Just do not abbreviate them as it really serves no purpose, and makes a lot of pain for people besides yourself maintaining code.

You can read about camel case at wikipedia.

The Source
For this program, I copied our HelloWorld.cs, and just modified it. I also have a modified makefile.

dataTypes.cs

using System;

// We create a class to contain it
class dataTypes
{
    static void Main(string[] args)
    {
        // The type 'bool' is always true or false
        bool isMonoTheOnlyPatentRisk = false;

        // To check if it is true or false in program code,
        // we can use an 'if statement'
        if(isMonoTheOnlyPatentRisk)
        {
            Console.WriteLine("isMonoTheOnlyPatentRisk was set to true");
        } else
        {
            Console.WriteLine("isMonoTheOnlyPatentRisk was set to false");
        }

        // Note how I named the variables -
        // lowercaseFirstThenCapitalizeEachWorld
        // this is called camel case -- I first read about it
        // in a Charles Petzold book, but I am sure it
        // probably didn't originate there.  Either way its
        // how I code, I recommend you name variables
        // as descriptive as possible. 

        // Note to only kids:  You will have the initial
        // reaction of saying: Why should I name stuff long?
        // That takes time and is stupid!
        // I used to be you 10 years ago, the problem is if
        // you need to use your code in several years of not
        // using it.  Your time will be completely wasted
        // figuring out what the heck is going on, despite
        // seeming so intuitive / easy to understand when you
        // did it

        // Integers store numbers
        int ourNumber = 5;

        // In most langauges we can evaluate these in
        // 'if statements' like bools above, but if we try in
        // C# this will fail.  In C# a bool can only be 0 for
        // false, 1 for true, or we can use the
        // Convert.ToBoolean function like this:
        if(Convert.ToBoolean(ourNumber))
        {
            Console.WriteLine("ourNumber was not 0!");
        } else
        {
            Console.WriteLine("ourNumber was 0");
        }

        // Lets see if we can compare this number
        if(ourNumber > 4)
        {
            Console.WriteLine("ourNumber was greater than 4");
        } else
        {
            Console.WriteLine("ourNumber was less than or equal to 4");
        }   

        // Make a string, and output it
        string helloWorldVariable = "Hello World!";
        Console.WriteLine(helloWorldVariable);
    }
}

Makefile

COMPILER=gmcs

all: helloWorld.exe dataTypes.exe

helloWorld.exe: helloWorld.cs
	$(COMPILER) helloWorld.cs

dataTypes.exe: dataTypes.cs
	$(COMPILER) dataTypes.cs
clean:
	rm -f helloWorld.exe dataTypes.exe

.PHONY: all clean

Output from our terminal

[sharms@sparrow mono]$ make
gmcs helloWorld.cs
gmcs dataTypes.cs
[sharms@sparrow mono]$ mono dataTypes.exe
isMonoTheOnlyPatentRisk was set to false
ourNumber was not 0!
ourNumber was greater than 4
Hello World!

Bonus Tip
Did you notice in the first part the way I named the bool? If you are programming, generally, your bool’s should start with ‘is’. This becomes very important once you start making complex ‘if statements’. Trust me on this one, it’s a habit you want.

Tags: , , ,

Python threads

Frequently I need to launch sub processes from Python. Sometimes these processes don’t work as expected, and end up blocking the program. If you want to launch a command from python, and cut it off after a certain amount of time, I recommend the threading module.

Here is how to limit the command to running for roughly 5 seconds:

#!/usr/bin/env python
import commands
import datetime
import time
import threading

class runCommand(threading.Thread):
    def run(self):
        status, output = commands.getstatusoutput('sleep 8')

if __name__ == "__main__":
    startTime = datetime.datetime.now()
    maximumRunTime = datetime.timedelta(seconds = 5)
    mythread = runCommand()
    mythread.start()

    while 1:
        if mythread.isAlive():
            print "Thread has not returned yet..."
        else:
            print "Thread is all done."
            break

        if (datetime.datetime.now() - startTime) > maximumRunTime:
            print "Thread has ran too long, giving up."
            break

        time.sleep(1)

    print "Program done"

So when we enter main, I assign the current time to startTime. Then I define maximumRunTime as a difference of 5 seconds. The thread is started, and every second we check to see if it is still running. If it is, we go ahead and check how much real time has elapsed.

Tags: , ,

Python-RPM

If you ever get to use this library, it’s design is horrible and it’s documentation is worse. If you want to retrieve the basic RPM data, I have a code snippet you can use:

python-rpm

As long as at the top of your file you had import rpm that should work file. That gives a few basic fields. To get a full list of fields, I actually used a reference from a perl module: http://cpansearch.perl.org/src/RJRAY/Perl-RPM-1.51/RPM/Constants.pm.

If you expect more information, it really isn’t out there. Every mailing list post tells you to visit slides at people.redhat.com, on a webpage which no longer exists from 2004. I was able to find it again, so this may help you also: http://www.ukuug.org/events/linux2004/programme/paper-PNasrat-1/rpm-python-slides/toc.html. ActiveState’s website also has a snippet showing a bit of what is contained in that hdr class, which really helped trying to decipher this: http://code.activestate.com/recipes/576767/.

And for all future python programmers out there, THERE IS NO NEED TO ABBREVIATE VARIABLE NAMES. We have big computers, lots of memory, and naming things ‘hdr’ and ‘ts’ help nobody, and make your code horrible for others to use. Seriously, expand them out.

Tags: , ,

Creating graphs in python

Slow Friday today, so was digging through my $HOME and found some graphs I was generating a few months ago. I wanted to create some graphs showing the number of connections to a server, and stumbled on CairoPlot.

Show me pretty pictures
Graph generated by CairoPlot

Unfortunately, I believe I hacked my CairoPlot.py file to make the dots and pulled some stuff from other repositories, so the code might not give the same results. You may want to try the trunk. But the general point still exists that it is really easy to use, and the library is really easy to hack. To make the graph above I just ran:


#!/usr/bin/python
colors = [ (0.2, .3, .65), (0.5, 0.7, .1), (.35, .2, .45), ]
graphData = {}
graphData['server1'] = [ 20, 12, 42, 14, 11, 35 ]
graphData['server2'] = [ 18, 23, 10, 17, 23, 25 ]
CairoPlot.dot_line_plot("./graphs/blog", graphData, 500, 500, axis = True, grid = True, dots = True, series_colors = colors)

Obviously this is really easy to script, so you can parse your syslog files, append them to the graphData dictionary corresponding to the server, and bam you have a full report of everything happening etc. What I did was use the datetime module to sort events into time buckets that were then used as graphs, giving a view of 24 hours or whatever the period entered was.

Parsing syslog
I will give you a hand here too. In python, to parse syslog, I used a module called pyparsing. It uses a parsing language which is pretty easy to understand if you give it 20 minutes or so. Ie to parse the syslog lines I was looking for, I did the following:


month = Word(string.uppercase, string.lowercase, exact=3)
integer = Word( nums )
ipAddress = delimitedList( integer, ".", combine=True )
serverDateTime = Combine( month + " " + integer + " " + integer + ":" + integer + ":" + integer )
hostname = Word( alphas + nums + "_" + "-" )
daemon = Word(alphas) + Suppress("[") + integer + Suppress("]:")
ip = Suppress("remote IP address ") + ipAddress
bnf = serverDateTime + ipAddress + daemon + ip

for line in syslogFile:
try:
fields = bnf.parseString(line)
...

Keep in mind I did this very quick, so I am sure it can be refactored a bunch, just an example.

Tags: , , ,

Fixing configuration files

I read on twitter from Ryan Gordon:

” So I might find I have a tree like this:

/home/icculus/Configs/Subversion
/home/icculus/Configs/DBUS/Config
/home/icculus/Configs/DBUS/Keyring
/home/icculus/Configs/Mozilla Thunderbird/Address Book
/home/icculus/Configs/Mozilla Thunderbird/Extensions

We can call it “Configs” or whatever, but it should probably be user-visible
(that is, without a prepended ‘.’), be standardized, and allow apps all the
space they want in their own folder under this root.”

I encourage you to read his whole post as that is just a little bit of it. He posted a follow up today:

“There is apparently a freedesktop thing, but looking aside the infuriating need to stick “XDG” on everything, I’m not sure I dig on the idea of using environment variables. I’d rather this have to call a piece of system-controlled code that can set policy, and handle things like creating directories, etc, so every app that needs to be changed needs extremely minimal changes, and reduce their ability to get things wrong. Granted, that system code could respect these XDG_* variables if that makes sense. Even looking beside that: dude, it’s 2009, can we stop using environment variables and shell scripts for everything? Is it heresy to ask that?

Honest to god, though: how’d that $BROWSER thing work out?”

I totally agree with him on this. The number of hidden files in my home directory is unmanagable, ugly, and a ton are not even close to intuively named.

To see how many hidden configuration files you have in your home directory you can run this:

ls -la ~ | awk '{print $8}' | egrep '^\.' | wc -l

I encourage anyone written an app to not put all of your configuration stuff in the base home directory, hopefully everyone can arrive at an intelligent solution. Ryan has a great start.

Tags: ,

Quick tip: Counting lines of code

I wanted to count the lines of code on a project I am working on. From the base directory of the project, I wanted to line count all files ending with .php. I also needed to ignore the directories with test, model/om, cache and plugins. Using grep, find and wc you can do this:


#!/bin/bash
grep -v '^ *$' `find . -iname "*.php" | grep -v model\/om | grep -v \/test | grep -v \/cache | grep -v \/plugins | grep -v \/web | grep -v lib\/OFC` | wc -l

I am at 15645 lines of code, which includes comments. What are you at?

Tags: , , ,