Category — Development Techniques

I haven’t had time to test most of these, but here are some ideas for installing Subversion and Trac:

• Read the manuals carefully and do a manual install. In my experience, Subversion isn’t too hard, but getting Trac working can take some tinkering.
  • The Subversion and TortoiseSVN manuals are excellent (if you’re a Windows user, you may find the TortoiseSVN manual more useful, since the Subversion manual is all command line).
  • I’ve found the Trac documentation a little harder (more spread-out, probably because it’s all wiki pags), but for Windows the key is the TracOnWindows page. I’ve found it useful to browse the TitleIndex list – you can find some stuff you’d miss otherwise.
• Try installing Trac 0.11 beta – it’s supposedly simpler to install (I haven’t tried yet).
• Try the TOW (TracOnWindows) project installer. I haven’t tried it yet; on the plus side, it’s a single installer for Trac, Subversion, and dependencies, and it’s been updated recently so the versions are current. OTOH, it appears to want to install everything in a fixed location (C:\TOW).
• You can try a VMWare Appliance (using the free VMWare Server) with Subversion and Trac already installed. I’ve looked at this; most appear to be somewhat out of date (e.g. Trac 0.9x), and for licensing reasons always use a open source OS (Linux, BSD) which might not be the best choice for everyone. Similar appliances might exist for the competition.
• You can use a hosting service with an installer (such as Webfaction) – that really does make installation easy, but upgrading can take a bit of work.
• It’s worth considering installing to a virtual machine (whether on Linux, Windows, etc) so you can move the Trac server around, or just to experiment without installing lots of programs on the host OS.

Finally, it’s always important to setup the server correctly, with the desired access rights and user log-ins.

Comments 4/23/2011: the TracOnWindows installer hasn’t been updated in a long time.

I still really like the VM approach; VirtualBox is another option (with images available), but check the licensing terms (using VirtualBox as a server might require a commercial license).

VisualSvn Server makes installing Subversion on Windows a breeze.  Trac can still be tricky.

Tony

I’ve added the desvn.py tool to remove Subversion’s .svn directories from a directory tree. It’s written in Python (and requires Python to run); it only took about an hour to write and test.

Tony

Well, actually some Subversion/TortoiseSVN/Trac hints.

I find the TortoiseSVN repository brower clunky. In fact, I often find it quicker when checking out a repository to find the path first with the svn web interface, and then cut and paste the path into the TortoiseSVN SVN Checkout dialog.

For browsing source, I normally use Trac’s Browse Source instead of the svn web interface. It has syntax highlighting for many programming languages and it’s easy to see the differences between revisions. If I need a quick look at code that I haven’t checked out (different project or older revision) on computer I’m using, I use Trac.

One neutral feature is that svn stores its information in hidden .svn directories. So, if you copy a svn working project from computer A to computer B, svn knows right away where the repository is, etc. If you delete the .svn directories, the project is effectively no longer under version control.

But, if you have a large directory tree, and need to get rid of the subversion information (to package up for an installer, etc), then all those .svn directories are a pain. I wrote a Python script to recurse through all the project subdirectories and remove them.

Comments 4/23/2011: the current TortoiseSvn repository browser seems much faster.

Tony

JavaWorld has a hands-on overview of CVS, Subversion, Mercurial, and Bazaar here.

Note 4/21/2011:  I’d say the most popular open source VCS’s are:

• For centralized, Subversion
• For distributed (DVCS), git (most popular) and Mercurial.

Tony

Version Control and Software Configuration Management Resources

In the end, good developers and good processes matter more than the tools. So here are two good sources for information and answers on version control and software configuration management: comp.software.config-mgmt newsgroup (low level of traffic, but I’ve seen some good conversations there) and CMCrossroads.

Commerical Version Control Software

A few years ago, I did an extensive look at commercial version control software. The two I was most impressed with were Perforce and AccuRev. Perforce is a traditional VCS, with a reputation for speed and good support. AccuRev has an innovative approach (streams) . Both are worth consideration, but are $750/developer or more, plus yearly maintenance fees. At work, I’ve been very happy with Subversion, but Larry O’Brian has seen some speed bumps.

Subversion

I’ve been using Subversion, and have been very happy with it. I still think it is an excellent version control system for most automation companies, and I will be doing a series of blog posts using it. Some of its good points:

• It’s free (and open source)
• Its centralized approach fits the model of most automation developers (unlike open source projects, which tend to be highly distributed).
• It runs well on Windows (unlike some open source version control systems)
• It has good, free documentation
• It is widely available for hosting (for example, it’s the only VCS available on Webfaction’s Control Panel)
• It has excellent tool support, including on Windows
• It is constantly updated (V1.5 will have substantial improvements)

Innovative Version Control Systems

The open source version control field has been very fertile, with several innovative approaches. I’d say the open source side has been more innovative than the commercial side. The ones that have caught my attention are git (of Linux fame, and right now probably the “hot” one), darcs, mercurial, and bazaar-ng. All of these systems are designed for distributed development, unlike most version control which is based around a central server. Later I plan on looking into them in-depth, and trying one out – I have a few project ideas that would benefit from a more distributed approach.

Tony

Webfaction dramatically improved their hosting plans, making it easy for me to add more applications.

So I’ve added a Subversion (svn) repository and a Trac site to this site. Right now they are pretty empty, but I will be adding to them over time.

The svn repository will host my blog project files, and will be used for posts on using version control. The Trac site contains additional information related to this blog. Both sites are read-only – I don’t have time to deal with link spam, wiki spam, or polluted repositories.

The svn site is http://svn.factoryswblog.org It will redirect a web browser from http, but not a client (such as my choice, TortoiseSVN ).

The trac site is http://trac.factoryswblog.org It will redirect from http.

Note that since I’m not paying extra for my own SSL certificate, you’re going to have to trust me (accept the browser pop-ups) if you want to use the sites.

Note 4/20/2011: my svn and trac sites are http; https was a fun experiment, but not necessary for this site.

Tony

Regression testing tries to verify that software changes do not cause current functionality to fail. A few years ago I wrote software to do regression testing for jobs on a Cognex Insight vision system. Since I do not have a Cognex system at my desk, I cannot give all the juicy details, but I can give an outline.

Machine builders often build a machine to work on a small set of sample parts (say 50). But when it gets into production with many more parts, there are often problems because the production parts show wider variations than the samples (or the manufacturer has made changes).

It’s the same for machine vision – you have to pick one particular part to start designing your machine vision job. Suppose the job works well in production, but somebody wants better results – so you grab a new part and get to work, right? Well, if you’re not careful, you can end up with a vision job that works great for that new part, but not for a typical part, and thus end up worse than you started.

So what I did was save a whole bunch of pictures of good parts and bad parts from production runs, then after I made any changes to the vision job, I ran my part database through the camera, and checked the results.

The Insight cameras use some sort of PowerPC processor, have Ethernet, RS-232 serial, and a bit of digital I/O (e.g. for trigger input). The Insight Explorer user interface software runs on a PC, is written in Java (and works pretty well; note that some newer Cognex products use the .NET framework), and uses a spreadsheet approach to machine vision, which has its pluses (such as simplicity) and minuses (like trying to sequence actions).

I like having the cameras on the network; you can work at your desk with the camera mounted far away. Cognex makes it especially nice by using standard Internet protocols, such as ftp to load and save jobs and pictures (BTW, if you need a free Windows ftp server or client, you should look at FileZilla), and telnet to control the camera.

Using telnet is basically like having a command line interface to the camera. You can do a lot with the camera (load jobs, trigger the camera, insert data, get results, etc), and it’s easy to test out ideas at the telnet command line, then codify them into a program.

I used Python with a free, open source telnet library. At least at the time, there was no free telnet library for .NET, and it didn’t make sense to buy one for this simple application. Then I wrote a Python module to do all the camera control I needed.

To do the regression testing, I wrote a Python program that loaded the desired vision job, then went through the database of pictures, loading each picture, triggering the camera (so it would run the job on the loaded picture), recording the result, comparing the result to the desired result, and then scoring the overall results.

Theoretically, it would be possible to do the same thing using the Insight Explorer software without using a real camera. In that case, you would use a GUI functional testing library (and some good free ones exist for Python) to load the jobs and pictures, then check the results. However, since Insight Explorer is written in Java, the normal GUI testing tools did not work (Java visual widgets aren’t the same as the native Windows ones – OK, maybe if you’re using SWT, but I think they were using Swing). I had limited success automatically inserting keystrokes and waiting, but it wasn’t reliable.

Tony

What am I talking about? Using a portable 2.5″ SATA drive to store and run virtual machine images. I use the free VMWare Server, and install it on the main machines I use. I keep virtual machine images on my portable drive, and run them from the drive, instead of copying (which takes a long time).

The portable drive lets me take my PC environment with me – for example, it allows me to work at home, and conveniently keep my work life separate from my personal life.

For automation projects, of course, PC virtual machines have their limitations – they don’t simulate robots, PLC’s, or a lot of other hardware. But they can still be very useful. For example, I have a project with two similar, but different (for different hardware) sets of COM objects. They can’t both be installed at the same time (they don’t meet the requirements for side by side or registration free installation), but I can’t compile in Visual Studio unless they are registered. But I can have two different virtual machines, each one with the appropriate set of objects registered.

What I did was combine a Hitachi 7K200 7200 RPM 16M byte buffer SATA 2.5″ hard drive – currently the fastest laptop drive (see StorageReview; I bought the 100G model from ZipZoomFly) – with an eSATA/USB case (I bought a Coolmax from Fry’s for $10 after MIR; other choices include Vantec). The USB connection provides power without wall warts and the ability to connect on most PC’s, the eSATA gives high speed (about 2-3x faster than my Acomdata 80G 2.5″ drive).

I’ve found that 3.5″ external HDD’s are fine for backup, but simply too big and clumsy to move around frequently (especially with the typical external power supply and cables). Microdrives are too slow for running a VM (I’ve tried on a 8G Memorex USB drive). 1.8″ drives are interesting and would probably work OK for VM’s, but are pricer and slower than my approach. Flash memory is great for transferring data, but I don’t trust it for running VM images, because of its write cycle limit (and flash isn’t so great at small, random writes).

Standard 2.5″ portable USB drives work OK for running VM images (I’ve used the Acomdata quite a bit), but the eSATA approach gives you roughly desktop HDD speed for a bit more money. I haven’t seen any commercial eSATA 2.5″ portable drives; for that matter, I don’t know of any commercial 7200 RPM 2.5″ USB drives, so right now you have to build your own – but it’s extremely easy.

Notes:

• On Windows, I highly recommend formatting the drive using NTFS; otherwise you’ll have to have VMWare split the drive image to deal with FAT’s 2G maximum file size. All NTFS drives (USB, Firewire, eSATA) have to be stopped (e.g. via the Safely Remove Hardware icon) before they can be safely removed.
• Virtual machines love memory – on the host PC, 1G is about the minimum, 2G is much better.
• The Coolmax case quality isn’t as good as my commercial Acomdata; the drive can wiggle around. I also like Acomdata’s brushed aluminum finish better.
• 7200RPM 2.5″ drives probably won’t work with just 1 USB port for power. A USB port can provide up to 2.5W (500 mA at 5V); the 7200RPM drives typically specify at least that much power to operate. Coolmax provided a single USB port to power plug cable and a dual USB (one power, one power & communications) to 5-pin cable. I plug in the eSATA cable first, and then the USB cable so the drive always starts up in eSATA mode.
• The Hitachi drive runs a little warm, but not hot. The Acomdata drive doesn’t even get warm.
• The eSATA removal procedure is a little more involved than on USB drives. One approach is to use the hardware device manager (e.g. provide convenient link to devmgmt.msc). Sometimes SATA drives (including internal ones) show up on the Windows “Safely Remove Hardware” icon. If they don’t, sometimes the Hotswap! utility can help (it does not support all SATA controllers).
• Frequently adding/removing eSATA drives may cause problems with software activation schemes. I suspect it may be more of a problem when Windows thinks it’s a permanent (instead of removable) drive.
• The eSATA/SATA standards have their quirks. Many early add-on cards do not support hot swap; probably most early motherboards do not either. SATA is point to point, so if your motherboard comes with only 2 SATA connectors, and you’ve already got two HDD’s, you’re going to have to add an eSATA card. SATA and eSATA connectors are slightly different. The eSATA cable is better than IDE, but not as flexible (or long) as USB, although I found a 6 ft model which seems more flexible. eSATA cables seem to be $10 or more.
• Since one PC needed more SATA ports, I bought a Vantec UGT-ST300 eSATA PCI card for $30. It was the cheapest card the got good reviews and guaranteed being able to hot swap. I haven’t had any problems with it yet.
• On the other computer, I have a bracket that converts a SATA cable (from the motherboard) to an eSATA connector.
• Since eSATA is not common, it’s best if you only use a few computers regularly with the portable HDD. And it’s convenient to pre-stage the cables – have a eSATA cable and USB power cable already plugged in to the computers you use the most (and carry an extra dual USB cable with the drive for other computers).

Note 4/19/2011: right now I’m basically running everything on my laptop.  If I wanted killer portable storage now, I’d look at combining a superfast SSD in a 1.8″ case with a USB 3.0 interface (if such a thing exists).  Maybe a USB 3.0 memory stick would work well, too.  But don’t forget to back up — flash mass storage can fail catastrophically too.

Tony

OK, this isn’t just for VB6, but I needed to call a function in a C DLL from VB6, and VB6 can only call functions that use the stdcall calling convention.

A properly written C DLL is easy to use from VB6, but a C DLL not written with VB6 in mind may require an adapter layer. C++ DLLs are typically very difficult to use from another compiler (or VB), due to incompatibilities in areas such as name mangling. Of course, COM DLLs and .NET assembly DLLs are totally different cases. COM objects are almost always easy to call from VB6. .NET assemblies are like C DLL’s: when properly designed with a COM interop, using them is like using a COM object, but if they’re not designed with COM in mind, then they can’t be used.

If you have the source or header files, you should be able to determine the calling convention – although you may have to know a bit about the compiler, too (so you know what the default convention is, and how each convention is identified).

A second method that sometimes works is to use Microsoft’s dumpbin program. You can get dumpbin for free by downloading and registering Visual C++ 2005 Express Edition (notes on Win32 development here) . By dumping the Export list and looking at the decorated names, you can determine the calling convention. I got the idea for this approach from here.

For a function defined as void foo(int a), its decorated name would be:

• _foo for __cdecl (C calling convention)
• _foo@4 for __stdcall (Windows calling convention)
• @foo@4 for __fastcall

I used dumpbin /EXPORTS path_to_dll from the Visual C++ 2005EE command prompt. However, not all DLLs export the decorated names, even for DLLs created with MS VC++. I’ve seen some that do and some that don’t; it might have to do with how the exported names are declared (e.g. in a DEF file). Non-Microsoft C compilers (MingW, gcc, Borland, etc) might decorate names differently.

By the way, you can use dumpbin /DEPENDENTS path_to_DLL to see what DLLs a DLL requires.

Another way is to use Python and the ctypes module. The ctypes module is included in Python 2.5; it’s an add-on to earlier versions. ctypes can load a DLL as either a Windows (stdcall) DLL using windll.LoadLibrary or a C (cdecl) DLL using cdll.LoadLibrary.

Here’s an example using Python and two functions in the MEI PC/DSP 2.5.09 MEDVC60F.DLL. One function (dsp_init) is stdcall; the other function (find_pci_dsp) is cdecl.

>>> from ctypes import *
>>> stdcall = windll.LoadLibrary(‘C:\\medvc60f.dll’)
>>> cdecl = cdll.LoadLibrary(‘C:\\medvc60f.dll’)
>>> ShortArray20Type = c_short * 20
>>> addr = ShortArray20Type()
>>> irq = ShortArray20Type()
>>> numBoards = c_short(0)
>>> cdecl.find_pci_dsp(byref(numBoards), byref(addr), byref(irq))
0
>>> stdcall.find_pci_dsp(byref(numBoards), byref(addr), byref(irq))
Traceback (most recent call last):
File “<interactive input>”, line 1, in <module>
ValueError: Procedure probably called with too many arguments (12 bytes in excess)
>>> cdecl.dsp_init(0×300)
Traceback (most recent call last):
File “<interactive input>”, line 1, in <module>
ValueError: Procedure called with not enough arguments (4 bytes missing) or wrong calling convention
>>> stdcall.dsp_init(0×300)
2

even if you’re using C# or VB.NET. How? By using Boo’s interpreter to try out ideas, test usage of .NET framework functions, and interact with components (COM, .NET). Then when you’re comfortable, you can write the polished code in the language of your choice.

This is one use for mixed language programming, mentioned by me here, but where the final software might not use both languages.

In fact, any .NET interpreter such as IronPython or IronRuby could be used. And it’s a great technique for other than .NET – I’ve used it a lot with Python on Win32, and you can use it in Java with JRuby, Jython, Groovy, etc. This approach could be very useful for embedded and factory software development.

On .NET IronPython and IronRuby have advantages because they’re official Microsoft languages (and IP has a book coming out – IronPython In Action) and the languages are already in wide use (Win32, *nix, Java). I’m using Boo because it comes with the open source SharpDevelop IDE, and right now it integrates better with .NET (IP does not support attributes well, and it’s not easy to make an IP assembly callable by other .NET languages, etc).

Short example – interactively using DirectoryInfo in Boo (text I typed in bold):

>>> di = DirectoryInfo(“C:\\Download”)
C:\Download
>>> files = di.GetFiles()
(AdbeRdr80_en_US.exe, AdbeRdr80_en_US_Nosso_error.log, Firefox Setup 2.0.0.1.exe, SharpDevelop_2.2.1.2648_Setup.msi, TortoiseSVN-1.4.3.8645-win32-svn-1.4.3.msi)
>>> files = di.GetFiles(‘t*’)
(TortoiseSVN-1.4.3.8645-win32-svn-1.4.3.msi)

Interactively calling a COM object in Boo:

>>> ieType = System.Type.GetTypeFromProgID(‘InternetExplorer.Application’)
System.__ComObject
>>> ie = System.Activator.CreateInstance(ieType)
System.__ComObject
>>> ie.Visible
false
>>> ie.Visible = true
true
>>> ie.Visible
true

So far I haven’t had to use InvokeMember to call COM functions. At least for C# and VB.NET (and probably Boo) it appears you need to use it if you are using late binding. BTW, it is possible to use late binding with COM events, but it is significant extra work.

Tony