Category — Development Techniques

I highly recommend good version control software for all automation developers, even for companies with just one developer.  I think for most automation companies the requirements are similar to these:

• Runs well on Windows (most automation vendors use only Windows)
• Easy to install
• Good documentation
• Easy to learn, including easy to learn concepts and how to handle typical version control situations
• Good tools support such as GUI clients, support for Visual Studio and/or Eclipse, web interfaces, and links to bug tracking systems
• Software is mature and reliable
• Available support
• Distributed team support is not needed

I recommend looking at Subversion first.  It meets all the requirements.  It runs well on Windows, is easy to install, has excellent documentation, and is widely used.  Tool support is excellent; highlights include the famous TortoiseSvn GUI client and the Trac light weight project management and bug tracking software.  Best of all, Subversion (like all the other projects mentioned in this post) is open source, so there are no license hassles, no limitations on named users, and no yearly maintenance fees.

Trac, on the other hand, isn’t so easy to install.  I’ve covered easier ways to install Subversion and Trac before .  Another approach I am looking at is running Subversion and Trac under XAMPP, the portable Apache/MySQL stack.  If it’s easy to do, this might be the best approach.

Distributed version control systems are definitely taking off.  Git seems to be in the lead, with Mercurial second.  They do have some major advantages over Subversion (which I hope to cover in detail sometime this year).  You should more concerned with which system works best for you, not what other developers (who probably have very different needs) are using.

Git normally isn’t the best choice for automation developers.  It doesn’t run well on Windows (you have to use cygwin (which is like running Linux) or msysgit (which is incomplete and poorly documented)), is still lacking in tool support, and appears to be harder to learn than Mercurial or Subversion.

Git does have some advantages; for example, it can reliably track CVS or Subversion repositories, and make a Subversion repository track a git one.  I will be using git for a project that requires creating a Subversion repository that tracks a CVS repository (and having a git repository that tracks the CVS repository is a nice extra).

Fossil is an interesting new distributed version control system with integrated bug tracking and wiki (similar to combining Mercurial with Trac).  It doesn’t have the tool support, documentation,  and widespread use of git, Mercurial, or Subversion, but it is much smaller (a single small executable), runs well under Windows, and has a GUI available via the browser (Web 2.0!).

I’d love to try out and do a detailed comparison of Subversion, git, Mercurial, and Fossil, but my realistic plan is:

1. Get Subversion and Trac running under XAMPP
2. Get git running for the CVS repository tracking project

and then, of course, write some posts about my experiences.

I was doing some debugging recently on an inherited Visual Basic.NET codebase. Something odd was going on – the code that reads a PLC’s input was always returning True, but I could see the input changing. The problem? The original programmer, who I’ll call “Bugs Bunny”, didn’t understand exception handling at all.

The function, part of a class that communicated with a PLC, went roughly like this:

Function bubba() As Boolean
    bubba = True
    Try
        bubba = getResultFromPlc()
    Catch ex As Exception
        logException()
    End Try
End Function

Since all Exceptions are caught and not handled any Exceptions raised in bubba()’s Try block just disappear. So all the code calling bubba() has no way of knowing if there are problem (such as the PLC isn’t powered or the serial port isn’t connected) . Instead, because the return value is initially set to True (bubba = True), if an Exception occurs, bubba() always return True.

Sometimes you do need to ignore Exceptions. For example, on a maintenance screen that repeatedly reads a PLC’s inputs, it’s good to indicate if there are any problems reading the inputs, but it’s not helpful to pop up a dialog box or exit the maintenance screen. Exceptions should be ignored occasionally as needed in the calling code, not completely swallowed up in the base library code.

Another common anti-pattern in Bug’s code is:

Sub joe()
    Try
        blah()
        blahblah()
    Catch ex As Exception
        logException()
        Throw ex
    End Try
End Sub
 
Sub groovy()
    Try
        joe()
    Catch ex As Exception
        logException()
        Throw ex
    End Try
End Sub

What’s the point? All that happens is the exception gets logged multiple times (adding confusion to the log file), more useless code is added (making the code harder to understand), and the program runs slower (throw, catching, and re-throwing Exceptions takes time).

Tony

I haven’t had time to test most of these, but here are some ideas for installing Subversion and Trac:

• Read the manuals carefully and do a manual install. In my experience, Subversion isn’t too hard, but getting Trac working can take some tinkering.
  • The Subversion and TortoiseSVN manuals are excellent (if you’re a Windows user, you may find the TortoiseSVN manual more useful, since the Subversion manual is all command line).
  • I’ve found the Trac documentation a little harder (more spread-out, probably because it’s all wiki pags), but for Windows the key is the TracOnWindows page. I’ve found it useful to browse the TitleIndex list – you can find some stuff you’d miss otherwise.
• Try installing Trac 0.11 beta – it’s supposedly simpler to install (I haven’t tried yet).
• Try the TOW (TracOnWindows) project installer. I haven’t tried it yet; on the plus side, it’s a single installer for Trac, Subversion, and dependencies, and it’s been updated recently so the versions are current. OTOH, it appears to want to install everything in a fixed location (C:\TOW).
• You can try a VMWare Appliance (using the free VMWare Server) with Subversion and Trac already installed. I’ve looked at this; most appear to be somewhat out of date (e.g. Trac 0.9x), and for licensing reasons always use a open source OS (Linux, BSD) which might not be the best choice for everyone. Similar appliances might exist for the competition.
• You can use a hosting service with an installer (such as Webfaction) – that really does make installation easy, but upgrading can take a bit of work.
• It’s worth considering installing to a virtual machine (whether on Linux, Windows, etc) so you can move the Trac server around, or just to experiment without installing lots of programs on the host OS.

Finally, it’s always important to setup the server correctly, with the desired access rights and user log-ins.

Tony

I’ve added the desvn.py tool to remove Subversion’s .svn directories from a directory tree. It’s written in Python (and requires Python to run); it only took about an hour to write and test.

Tony

Well, actually some Subversion/TortoiseSVN/Trac hints.

I find the TortoiseSVN repository brower clunky. In fact, I often find it quicker when checking out a repository to find the path first with the svn web interface, and then cut and paste the path into the TortoiseSVN SVN Checkout dialog.

For browsing source, I normally use Trac’s Browse Source instead of the svn web interface. It has syntax highlighting for many programming languages and it’s easy to see the differences between revisions. If I need a quick look at code that I haven’t checked out (different project or older revision) on computer I’m using, I use Trac.

One neutral feature is that svn stores its information in hidden .svn directories. So, if you copy a svn working project from computer A to computer B, svn knows right away where the repository is, etc. If you delete the .svn directories, the project is effectively no longer under version control.

But, if you have a large directory tree, and need to get rid of the subversion information (to package up for an installer, etc), then all those .svn directories are a pain. I wrote a Python script to recurse through all the project subdirectories and remove them.

Tony

JavaWorld has a hands-on overview of CVS, Subversion, Mercurial, and Bazaar here.

Tony

Version Control and Software Configuration Management Resources

In the end, good developers and good processes matter more than the tools. So here are two good sources for information and answers on version control and software configuration management: comp.software.config-mgmt newsgroup (low level of traffic, but I’ve seen some good conversations there) and CMCrossroads.

Commerical Version Control Software

A few years ago, I did an extensive look at commercial version control software. The two I was most impressed with were Perforce and AccuRev. Perforce is a traditional VCS, with a reputation for speed and good support. AccuRev has an innovative approach (streams) . Both are worth consideration, but are $750/developer or more, plus yearly maintenance fees. At work, I’ve been very happy with Subversion, but Larry O’Brian has seen some speed bumps.

Subversion

I’ve been using Subversion, and have been very happy with it. I still think it is an excellent version control system for most automation companies, and I will be doing a series of blog posts using it. Some of its good points:

• It’s free (and open source)
• Its centralized approach fits the model of most automation developers (unlike open source projects, which tend to be highly distributed).
• It runs well on Windows (unlike some open source version control systems)
• It has good, free documentation
• It is widely available for hosting (for example, it’s the only VCS available on Webfaction’s Control Panel)
• It has excellent tool support, including on Windows
• It is constantly updated (V1.5 will have substantial improvements)

Innovative Version Control Systems

The open source version control field has been very fertile, with several innovative approaches. I’d say the open source side has been more innovative than the commercial side. The ones that have caught my attention are git (of Linux fame, and right now probably the “hot” one), darcs, mercurial, and bazaar-ng. All of these systems are designed for distributed development, unlike most version control which is based around a central server. Later I plan on looking into them in-depth, and trying one out – I have a few project ideas that would benefit from a more distributed approach.

Tony

Webfaction dramatically improved their hosting plans, making it easy for me to add more applications.

So I’ve added a Subversion (svn) repository and a Trac site to this site. Right now they are pretty empty, but I will be adding to them over time.

The svn repository will host my blog project files, and will be used for posts on using version control. The Trac site contains additional information related to this blog. Both sites are read-only – I don’t have time to deal with link spam, wiki spam, or polluted repositories.

The svn site is https://svn.factoryswblog.org It will redirect a web browser from http, but not a client (such as my choice, TortoiseSVN ).

The trac site is https://trac.factoryswblog.org It will redirect from http.

Note that since I’m not paying extra for my own SSL certificate, you’re going to have to trust me (accept the browser pop-ups) if you want to use the sites.

Tony

Regression testing tries to verify that software changes do not cause current functionality to fail. A few years ago I wrote software to do regression testing for jobs on a Cognex Insight vision system. Since I do not have a Cognex system at my desk, I cannot give all the juicy details, but I can give an outline.

Machine builders often build a machine to work on a small set of sample parts (say 50). But when it gets into production with many more parts, there are often problems because the production parts show wider variations than the samples (or the manufacturer has made changes).

It’s the same for machine vision – you have to pick one particular part to start designing your machine vision job. Suppose the job works well in production, but somebody wants better results – so you grab a new part and get to work, right? Well, if you’re not careful, you can end up with a vision job that works great for that new part, but not for a typical part, and thus end up worse than you started.

So what I did was save a whole bunch of pictures of good parts and bad parts from production runs, then after I made any changes to the vision job, I ran my part database through the camera, and checked the results.

The Insight cameras use some sort of PowerPC processor, have Ethernet, RS-232 serial, and a bit of digital I/O (e.g. for trigger input). The Insight Explorer user interface software runs on a PC, is written in Java (and works pretty well; note that some newer Cognex products use the .NET framework), and uses a spreadsheet approach to machine vision, which has its pluses (such as simplicity) and minuses (like trying to sequence actions).

I like having the cameras on the network; you can work at your desk with the camera mounted far away. Cognex makes it especially nice by using standard Internet protocols, such as ftp to load and save jobs and pictures (BTW, if you need a free Windows ftp server or client, you should look at FileZilla), and telnet to control the camera.

Using telnet is basically like having a command line interface to the camera. You can do a lot with the camera (load jobs, trigger the camera, insert data, get results, etc), and it’s easy to test out ideas at the telnet command line, then codify them into a program.

I used Python with a free, open source telnet library. At least at the time, there was no free telnet library for .NET, and it didn’t make sense to buy one for this simple application. Then I wrote a Python module to do all the camera control I needed.

To do the regression testing, I wrote a Python program that loaded the desired vision job, then went through the database of pictures, loading each picture, triggering the camera (so it would run the job on the loaded picture), recording the result, comparing the result to the desired result, and then scoring the overall results.

Theoretically, it would be possible to do the same thing using the Insight Explorer software without using a real camera. In that case, you would use a GUI functional testing library (and some good free ones exist for Python) to load the jobs and pictures, then check the results. However, since Insight Explorer is written in Java, the normal GUI testing tools did not work (Java visual widgets aren’t the same as the native Windows ones – OK, maybe if you’re using SWT, but I think they were using Swing). I had limited success automatically inserting keystrokes and waiting, but it wasn’t reliable.

Tony

What am I talking about? Using a portable 2.5″ SATA drive to store and run virtual machine images. I use the free VMWare Server, and install it on the main machines I use. I keep virtual machine images on my portable drive, and run them from the drive, instead of copying (which takes a long time).

The portable drive lets me take my PC environment with me – for example, it allows me to work at home, and conveniently keep my work life separate from my personal life.

For automation projects, of course, PC virtual machines have their limitations – they don’t simulate robots, PLC’s, or a lot of other hardware. But they can still be very useful. For example, I have a project with two similar, but different (for different hardware) sets of COM objects. They can’t both be installed at the same time (they don’t meet the requirements for side by side or registration free installation), but I can’t compile in Visual Studio unless they are registered. But I can have two different virtual machines, each one with the appropriate set of objects registered.

What I did was combine a Hitachi 7K200 7200 RPM 16M byte buffer SATA 2.5″ hard drive – currently the fastest laptop drive (see StorageReview; I bought the 100G model from ZipZoomFly) – with an eSATA/USB case (I bought a Coolmax from Fry’s for $10 after MIR; other choices include Vantec). The USB connection provides power without wall warts and the ability to connect on most PC’s, the eSATA gives high speed (about 2-3x faster than my Acomdata 80G 2.5″ drive).

I’ve found that 3.5″ external HDD’s are fine for backup, but simply too big and clumsy to move around frequently (especially with the typical external power supply and cables). Microdrives are too slow for running a VM (I’ve tried on a 8G Memorex USB drive). 1.8″ drives are interesting and would probably work OK for VM’s, but are pricer and slower than my approach. Flash memory is great for transferring data, but I don’t trust it for running VM images, because of its write cycle limit (and flash isn’t so great at small, random writes).

Standard 2.5″ portable USB drives work OK for running VM images (I’ve used the Acomdata quite a bit), but the eSATA approach gives you roughly desktop HDD speed for a bit more money. I haven’t seen any commercial eSATA 2.5″ portable drives; for that matter, I don’t know of any commercial 7200 RPM 2.5″ USB drives, so right now you have to build your own – but it’s extremely easy.

Notes:

• On Windows, I highly recommend formatting the drive using NTFS; otherwise you’ll have to have VMWare split the drive image to deal with FAT’s 2G maximum file size. All NTFS drives (USB, Firewire, eSATA) have to be stopped (e.g. via the Safely Remove Hardware icon) before they can be safely removed.
• Virtual machines love memory – on the host PC, 1G is about the minimum, 2G is much better.
• The Coolmax case quality isn’t as good as my commercial Acomdata; the drive can wiggle around. I also like Acomdata’s brushed aluminum finish better.
• 7200RPM 2.5″ drives probably won’t work with just 1 USB port for power. A USB port can provide up to 2.5W (500 mA at 5V); the 7200RPM drives typically specify at least that much power to operate. Coolmax provided a single USB port to power plug cable and a dual USB (one power, one power & communications) to 5-pin cable. I plug in the eSATA cable first, and then the USB cable so the drive always starts up in eSATA mode.
• The Hitachi drive runs a little warm, but not hot. The Acomdata drive doesn’t even get warm.
• The eSATA removal procedure is a little more involved than on USB drives. One approach is to use the hardware device manager (e.g. provide convenient link to devmgmt.msc). Sometimes SATA drives (including internal ones) show up on the Windows “Safely Remove Hardware” icon. If they don’t, sometimes the Hotswap! utility can help (it does not support all SATA controllers).
• Frequently adding/removing eSATA drives may cause problems with software activation schemes. I suspect it may be more of a problem when Windows thinks it’s a permanent (instead of removable) drive.
• The eSATA/SATA standards have their quirks. Many early add-on cards do not support hot swap; probably most early motherboards do not either. SATA is point to point, so if your motherboard comes with only 2 SATA connectors, and you’ve already got two HDD’s, you’re going to have to add an eSATA card. SATA and eSATA connectors are slightly different. The eSATA cable is better than IDE, but not as flexible (or long) as USB, although I found a 6 ft model which seems more flexible. eSATA cables seem to be $10 or more.
• Since one PC needed more SATA ports, I bought a Vantec UGT-ST300 eSATA PCI card for $30. It was the cheapest card the got good reviews and guaranteed being able to hot swap. I haven’t had any problems with it yet.
• On the other computer, I have a bracket that converts a SATA cable (from the motherboard) to an eSATA connector.
• Since eSATA is not common, it’s best if you only use a few computers regularly with the portable HDD. And it’s convenient to pre-stage the cables – have a eSATA cable and USB power cable already plugged in to the computers you use the most (and carry an extra dual USB cable with the drive for other computers).

Tony