‘ERROR: Garbage option.’ when using ps and awk

December 11th, 2009

The other day I wanted to create a simple bash script that would receive the name of a process and print out its pid. I wanted to run ps and then use awk to filter out the correct process and print out its pid. So I created the following bash script (called getPIDOf):

#! /bin/bash

command=”ps -o pid,command -C $1″
for (( i=1; i<=$#; i+=1 )); do
curval=${!i}
command=”$command | grep \”$curval\”"
done
command=”$command | awk ‘!/awk/ && !/getPIDOf/ {print \$1}’”

echo $command
$command

I was quite pleased with myself that I had even added in the option to specify multiple identifiers for the process. But when I ran the script I got the following output:

yuval@obt:$ ./getPIDOf nc
ps -o pid,command -C nc | grep “nc” | awk ‘!/awk/ && !/getPIDOf/ {print $1}’
ERROR: Garbage option.
********* simple selection *********  ********* selection by list *********
-A all processes                      -C by command name
-N negate selection                   -G by real group ID (supports names)
-a all w/ tty except session leaders  -U by real user ID (supports names)
-d all except session leaders         -g by session OR by effective group name
-e all processes                      -p by process ID
T  all processes on this terminal     -s processes in the sessions given
a  all w/ tty, including other users  -t by tty
g  OBSOLETE — DO NOT USE             -u by effective user ID (supports names)
r  only running processes             U  processes for specified users
x  processes w/o controlling ttys     t  by tty
*********** output format **********  *********** long options ***********
-o,o user-defined  -f full            –Group –User –pid –cols –ppid
-j,j job control   s  signal          –group –user –sid –rows –info
-O,O preloaded -o  v  virtual memory  –cumulative –format –deselect
-l,l long          u  user-oriented   –sort –tty –forest –version
-F   extra full    X  registers       –heading –no-heading –context
********* misc options *********
-V,V  show version      L  list format codes  f  ASCII art forest
-m,m,-L,-T,H  threads   S  children in sum    -y change -l format
-M,Z  security data     c  true command name  -c scheduling class
-w,w  wide output       n  numeric WCHAN,UID  -H process hierarchy

I thought that I must have something wrong with the escaping. But when I ran the command that was printed out, it worked fine:

yuval@obt:$ ps -o pid,command -C nc | grep “nc” | awk ‘!/awk/ && !/getPIDOf/ {print $1}’
892

I even tried the following simplified script:

#! /bin/bash

command=”ps -o pid,command -C $1″
command2=”awk ‘!/awk/ && !/getPIDOf/ {print \$1}’”

echo “$command | $command2″
$command | $command2

And got the following output:

yuval@obt:$ ./getPIDOf nc
ps -o pid,command -C nc | awk ‘!/awk/ && !/getPIDOf/ {print $1}’
awk: 1: unexpected character ”’

By this time I was quite puzzled. My simple getPIDOf script had become full of strange behaviors. So I decided to take the quick route out and use Perl instead of awk.

Here is my final version of the getPIDOf script:

#!/usr/bin/perl -w
use strict;

die “Usage: $0 <identifier1> [identifier2] …\n” unless (@ARGV);

my $command = “/bin/ps -o pid,command -C \”$ARGV[0]\”";
my $output = `$command`;
foreach my $line (split(/\n/, $output))
{
if ($line =~ /^\s*(\d+)\s+(.+?)$/)
{
my $pid = $1;
my $command = $2;
my $found_mismatch = 0;
foreach my $filter (@ARGV)
{
if ($command !~ /$filter/)
{
$found_mismatch = 1;
last;
}
}

if ($found_mismatch){next;
}

print “$pid\n”;
}
}

And the output, as desired, is:

yuval@obt:$ ./getPIDOf nc
892

Improved SWIG C# wrappers for std::vector and std::map

November 27th, 2009

My previous posts on this topic part 1 and part 2 were posted about half a year ago. Since then SWIG version 1.3.40 has been released and it includes several enhancements in the C# wrapper for std::vector and std::map based on the work that I did, but with several great improvements implemented by William Fulton, the person in charge of the C# wrappers in SWIG.

Since version 1.3.40 wrapping std::vector and std::map in C# is extremely simple. I’ll give a quick overview:

Wrapping std::vector

Lets assume you have a C++ class called MyClass with a method std::vector<int> GetIntVector(); You want to wrap MyClass from  C++ to C# and have the GetIntVector method return an IList<int> in C#. In order to do this, your swig interface file (the .i file) should look something like this:

/* File : MyProject.i */
%module MyProject

%{
#include “MyClass.h”
%}

%include “std_vector.i”
%include “MyClass.h”

%template(Int_Vector) std::vector<int>;

The C# class Int_Vector will implement the IList<int> interface as desired.

Now lets make things a bit trickier by introducing a new class called MyItem and lets add a method to MyClass with the following signature: std::vector<MyItem> GetItems(); In this case you’ll want the C# method to return an IList<MyItem>. One important issue that should be mentioned here is that any class being used in the IList<> generic must overload the == operator. Otherwise, it is only possible to use the IEnumerable<> generic interface.

So if MyItem doesn’t overload the == operator, and you’re content with having the C# method return an IEnumerable<MyItem>, then your swig interface file will look something like this:

/* File : MyProject.i */
%module MyProject

%{
#include “MyClass.h”
#include “MyItem.h”
%}

%include “std_vector.i”
%include “MyClass.h”
%include “MyItem.h”

%template(Int_Vector) std::vector<int>;
%template(Item_Vector) std::vector<MyItem>;

The C# class Item_Vector will implement the IEnumerable<MyItem> interface.

If, however, MyItem overloads the == operator, then you can have the C# Item_Vector implement the IList<MyItem> interface. You’ll need to tell swig that your class can be used with the IList<> interface by using the SWIG_STD_VECTOR_ENHANCED macro. Your swig interface file will look something like this:

/* File : MyProject.i */
%module MyProject

%{
#include “MyClass.h”
#include “MyItem.h”
%}

%include “std_vector.i”
%include “MyClass.h”
%include “MyItem.h”

%template(Int_Vector) std::vector<int>;
SWIG_STD_VECTOR_ENHANCED(MyItem)
%template(Item_Vector) std::vector<MyItem>;

Now the C# class Item_Vector will implement the IList<MyItem> interface.

Wrapping std::map

Thanks to some nifty features added to the SWIG core, it is now possible to wrap C++ std::maps into C# IDictionary<> generic classes very easily.

Lets assume you have a C++ class called MyClass with a method std::map<std::string, int> GetMap(); You want to wrap MyClass from  C++ to C# and have the GetMap method return an IDictionary<string,int> in C#. In order to do this, your swig interface file should look something like this:

/* File : MyProject.i */
%module MyProject

%{
#include “MyClass.h”
%}

%include “std_string.i”
%include “std_vector.i”
%include “MyClass.h”

%template(String_Int_Map) std::map<std::string, int>;

The C# class String_Int_Map will implement the IDictionary<string,int> interface as desired. It’s that simple. No need for any specialization macros or anything.

Offline installation of PEAR packages

November 26th, 2009

A few days ago I was trying to set up an offline installation of the FirePHPCore PEAR package on a computer that isn’t connected to the internet. This task wasn’t as easy as I thought it would be.

I downloaded the package (on a computer that has an internet connection) using:

  1. pear channel-discover pear.firephp.org
  2. pear download firephp/FirePHPCore

This gave me a file called FirePHPCore-0.3.1.tgz. Quite pleased with myself I transferred this file to the target machine and ran:

pear install -O FirePHPCore-0.3.1.tgz

This gave me the following error:

Parsing of package.xml from file “/tmp/pear/cache/package.xml” failed
Cannot download non-local package “FirePHPCore-0.3.1.tgz”
Package “FirePHPCore-0.3.1.tgz” is not valid
install failed

That can’t be good. I tried to fix this problem in several ways. I ended up with the following solution:

  1. Extract the contents of the tar file using: tar xzf FirePHPCore-0.3.1.tgz. This creates a file called package.xml and a directory called FirePHPCore-0.3.1 with some files in it.
  2. Edit the package.xml file and change the contents of the “channel” tag (line 4) from pear.firephp.org to pear.php.net
  3. Recreate the tgz file using: tar czf FirePHPCore-0.3.1.tgz package.xml FirePHPCore-0.3.1/*
  4. You are now the proud owner of a FirePHPCore-0.3.1.tgz file that can be installed offline.

Running pear install -O FirePHPCore-0.3.1.tgz on the new tgz file now results in:

install ok: channel://pear.php.net/FirePHPCore-0.3.1

Much better.

Building ACE on Windows using Visual Studio

November 16th, 2009

The Adaptive Communication Environment (ACE)  framework is an extensive set of cross-platform tools in C++. In this post I will describe in detail the steps required to set up the ACE framework on Windows using Visual Studio, and to set up a project using the ACE framework. The process is not complicated, but there are several pitfalls that one should be aware of.

Here we go!

Note: Steps 1-6 are partially covered in the ACE Build Guide. I give them here with some extra hints and tips.

  1. Download the ACE framework from here. We will be using the full version of ACE.zip (last line under “Latest Micro Release Kit”).
  2. Unzip this file into the directory where you want your ACE files to be located. I put my ACE files under C:\Programming\C++. Unzipping the file creates an ACE_Wrappers directory (C:\Programming\C++\ACE_Wrappers in my case). We’ll call this directory the ACE_ROOT directory.
  3. Go to the ACE_ROOT\ace directory (C:\Programming\C++\ACE_Wrappers\ace in my case) and create a file called “config.h”. Edit this file with notepad or any other text editor and put the following text in the file:
    #include “ace/config-win32.h”
  4. In the ACE_ROOT directory there are several solutions for different versions of Visual Studio and for different types of libraries. In this tutorial I will be using Visual C++ 2008 Express Edition (which is also called vc9) and building ACE as a static library. So the solution I  need is ‘ACE_wrappers_vc_9_Static.sln’. Select the solution that fits your needs.
  5. Select your configuration (Debug or Release) and build the ACE project in your solution. In my case this is ‘ACE_vc9_Static’.
    Building the ACE library
  6. After a few minutes of building, you should have the ACE library file in your ACE_ROOT\lib directory. I built using the Debug configuration, so the file that was created is c:\Programming\C++\ACE_Wrappers\lib\ACEsd.lib
  7. We are now ready to create our first ACE project! Create an empty C++ solution and then edit the project properties.
  8. In “Properties”–”C/C++”–”General”–”Additional Include Directories” add the ACE_ROOT directory.
    Additional Include Directories

  9. In “Properties”–”C/C++”–”Preprocessor”–”Preprocessor Definitions” set the following:
    ACE_AS_STATIC_LIBS
    _CRT_SECURE_NO_WARNINGS
    WIN32

    Preprocessor Definitions

  10. In “Properties”–”C/C++”–”Code Generation”–”Runtime Library” set to Multi-Threaded debug (/MTd) for the Debug configuration and Multi-Threaded (/MT) for the Release configuration.
    Code Generation
  11. In “Properties”–”Linker”–”General”–”Additional Library Directories” add the ACE_ROOT\lib directory.
    Additional Library Directories
  12. In “Properties”–”Linker”–”Input”–”Additional Dependencies” add ACEds.lib for the Debug configuration and ACEs.lib for the Release configuration.
    Linker Input
  13. Your project is now configured to work with ACE. You can create your project code and start using the ACE framework. Following is a simple example program that runs a thread to perform some work.
    ACE Test Program

That’s all folks!

Loading Perl Modules from a Dynamic or Unknown Location

November 13th, 2009

Perl modules are sometimes located in non-standard directories. There are several ways to use these modules:

  • Add the directory to the PERL5LIB environment variable.
  • Run the script using the perl -l.
  • Within the script ‘use lib‘ with the directory name.

These methods are quite common and handle most of the cases perfectly.

For example:

#!/usr/bin/perl -w
use strict;

use lib (”/home/users/yuval/perl/modules”);
use MyModule;

This works fine as long as you know that you need to search in /home/users/yuval/perl/modules to find MyModule.

But what happens if you don’t know the location of the module ahead of time?
None of the methods mentioned above can handle this situation. Consider the following code:

#!/usr/bin/perl -w
use strict;

my $user = GetCoolestUser();
use lib (”/home/users/$user/perl/modules”);
use MyModule;

sub GetCoolestUser()
{
return “yuval”;
}

Running this code gives a bunch of really scary compilation errors.

Yes – compilation errors. Yes – in Perl. What gives you ask?

The reason is that the Perl compiler tries to locate all of the modules that are going to be loaded before running the script. It does this before values are given to any variables, and before any functions are evaluated. Therefore the $user variable is undefined when it is evaluated in the parameters to the ‘use lib‘ statement, and then MyModule isn’t found in any of the directories that Perl searches.

There are two ways around this:

1. Push the directory name directly into the @INC array in the BEGIN function

#!/usr/bin/perl -w
use strict;

BEGIN
{
sub GetCoolestUser()
{
return “yuval”;
}

my $user = GetCoolestUser();
push(@INC, “/home/users/$user/perl/modules”);
}
use MyModule;

You may notice that this code doesn’t use the elegant ‘use lib‘ but rather the tasteless pushing of the directory into the @INC array. Also – the GetCoolestUser function has to be defined within the BEGIN block in order to be used.

2. Use the eval function

#!/usr/bin/perl -w
use strict;

my $user = GetCoolestUser();
eval “use lib qw(/home/users/$user/perl/modules);
use MyModule;”;
if ($@) {
die “Error loading module MyModule:\n$@\n”;
}

sub GetCoolestUser()
{
return “yuval”;
}

This method “tricks” Perl compiler and allows loading the module from any location you wish. This method is stronger than the previous method mentioned because the module loading occurs during run time and the location can be calculated dynamically during run time as well.

Visualgos – The 8-Puzzle

September 5th, 2009

I’ve just added a new problem visualization. The visualization is of the 8-Puzzle – a classic game in which you have a 3 by 3 board with tiles numbered 1 to 8, and one missing tile. Your goal is to bring the board to a state in which the tiles are ordered according to their numbers, and the empty spot is in the bottom right-hand corner. The only movements allowed are taking a tile that is adjacent to the empty spot and sliding it (horizontally or vertically) into the empty spot.

The visualizations added implement the DFS algorithm, the greedy algorithm and the A* algorithm.

You can find the visualization here, along with detailed information about the algorithms.

Introducing – Visualgos

July 15th, 2009

Visualgos is a collection of visualizations for algorithms written in Flash.

When I study a new algorithm I usually want to get a feel for it by seeing how it performs. It usually takes hours of programming and debugging in order to get something up and running. The idea of “Visualgos” is to supply a collection of visualizations for different algorithms so that anyone can get a feel for an algorithm without needing to spend hours coding it.

I’ve already added a visualization of a heuristic search algorithm in the form of a Tic-Tac-Toe game and a visualization of a heuristic repair algorithm in the form of an N Queens puzzle solver.

If you are interested in contributing to this project, please contact me at:

yuval (dot) baror [dot] website {at} gmail (dot) com

Superl

May 23rd, 2009

Superl is a project I developed several years ago. It was inspired by a great program I was working with that went by the same name.

Superl is a graphical Perl interpreter written in Perl. It opens one simple window in which you can type in your Perl commands and it will execute these commands for you. Superl also has some nifty features like:

  • Scrolling through the history of commands you’ve entered.
  • The _ function gets the result of the previous command (like ‘_’ in python).
  • Function/Variable completion with ctrl+space.
  • The result of each command is shown in the output window in blue.
  • Prints to STDOUT from your program will be shown in the output window in green.
  • Prints to STDERR from your program will be shown in the output window in red.

Superl runs on both Windows and Unix/Linux. All it needs is Perl and the Perl Tk package.

Superl is extremely useful for performing quick tasks with Perl or for checking out small pieces of code. Superl isn’t meant to be used as a Perl IDE. It doesn’t have any form of file management. It’s meant to be a graphical Perl interpreter.

I’ve registered Superl under Sourceforge as an open source project. It is distributed under a GPL license.

The Sourceforge project homepage is here. On it you can find more details about the project and you can download the project files.

The Game of Evolution

May 16th, 2009

I’ve recently added a new page to my website – The Game of Evolution.

This game is an advanced version of the famous Game of Life.  Similar to the Game of Life, the Game of Evolution is a zero player game. The game starts with an initial configuration and the user can watch how the game advances.

In this game animals must survive in a world according to a set of rules. Each animal starts with an initial amount of energy, and loses energy each turn when it performs actions like moving or reproducing. An animal can gain energy by performing actions like sleeping or eating. The animals make their decision as to which action to perform each turn according to their genome. Each animal has its own genome that is transferred to its offspring with a random mutation. The winner of the game is the animal which outlives all the other animals.

It is interesting to study which genomes are more likely to survive in the world, and how small parameters like the length of the genome, the amount of energy spent and gained by different actions and the initial placement of the animals affect the outcome of the game.

The game is located here.

More information on the game rules is located here.

The game is currently in a fairly initial state, so many more features are expected to be released in the future.

I’d be more than happy to hear any comments/ideas you may have. Email me at yuval (dot) baror [dot] website {at} gmail (dot) com

Improved SWIG C# wrappers std_vector.i and std_map.i – part 2

May 9th, 2009

* Note: since posting this a new version of SWIG has been released (version 1.3.40). This version includes the improved C# wrappers discussed here. A simple explanation about these wrappers can be found here.

After posting my previous post on the improved SWIG C# wrappers for std::vector (std_vector.i) and std::map (std_map.i) I was contacted by one of the main developers of SWIG, William Fulton. This led me to discover the extensive testing suite that SWIG is delivered with.

I tested my improved std_vector.i wrapper against the testing suite, and this went well. Then I wanted to test my std_map.i wrapper, as it is a complete rewrite of the existing wrapper.  Since there was no existing test suite for the std_map.i wrapper, I created my own test suite for this wrapper. This flushed out a lot of bugs, and the wrapper is hopefully in much better shape now.

Here is the new version of std_vector.i wrapper: std_vector.i

And the new version of the std_map.i wrapper: std_map.i

The two files relevant for the testing suite of std_map.i are: li_std_map.i and li_std_map_runme.cs

These can be incorporated in the existing SWIG testing suite. I tested them on Windows using Visual Studio. I’m also attaching my C++ and C# solutions for this testing suite. To use them – first rebuild the C++ solution, and then rebuild the C# solution and run it. std_map_test.zip

These wrappers will now hopefully be included in the next release of SWIG, though a good code review wouldn’t hurt…