Category — Development Techniques

What am I talking about? Using a portable 2.5″ SATA drive to store and run virtual machine images. I use the free VMWare Server, and install it on the main machines I use. I keep virtual machine images on my portable drive, and run them from the drive, instead of copying (which takes a long time).

The portable drive lets me take my PC environment with me – for example, it allows me to work at home, and conveniently keep my work life separate from my personal life.

For automation projects, of course, PC virtual machines have their limitations – they don’t simulate robots, PLC’s, or a lot of other hardware. But they can still be very useful. For example, I have a project with two similar, but different (for different hardware) sets of COM objects. They can’t both be installed at the same time (they don’t meet the requirements for side by side or registration free installation), but I can’t compile in Visual Studio unless they are registered. But I can have two different virtual machines, each one with the appropriate set of objects registered.

What I did was combine a Hitachi 7K200 7200 RPM 16M byte buffer SATA 2.5″ hard drive – currently the fastest laptop drive (see StorageReview; I bought the 100G model from ZipZoomFly) – with an eSATA/USB case (I bought a Coolmax from Fry’s for $10 after MIR; other choices include Vantec). The USB connection provides power without wall warts and the ability to connect on most PC’s, the eSATA gives high speed (about 2-3x faster than my Acomdata 80G 2.5″ drive).

I’ve found that 3.5″ external HDD’s are fine for backup, but simply too big and clumsy to move around frequently (especially with the typical external power supply and cables). Microdrives are too slow for running a VM (I’ve tried on a 8G Memorex USB drive). 1.8″ drives are interesting and would probably work OK for VM’s, but are pricer and slower than my approach. Flash memory is great for transferring data, but I don’t trust it for running VM images, because of its write cycle limit (and flash isn’t so great at small, random writes).

Standard 2.5″ portable USB drives work OK for running VM images (I’ve used the Acomdata quite a bit), but the eSATA approach gives you roughly desktop HDD speed for a bit more money. I haven’t seen any commercial eSATA 2.5″ portable drives; for that matter, I don’t know of any commercial 7200 RPM 2.5″ USB drives, so right now you have to build your own – but it’s extremely easy.

Notes:

• On Windows, I highly recommend formatting the drive using NTFS; otherwise you’ll have to have VMWare split the drive image to deal with FAT’s 2G maximum file size. All NTFS drives (USB, Firewire, eSATA) have to be stopped (e.g. via the Safely Remove Hardware icon) before they can be safely removed.
• Virtual machines love memory – on the host PC, 1G is about the minimum, 2G is much better.
• The Coolmax case quality isn’t as good as my commercial Acomdata; the drive can wiggle around. I also like Acomdata’s brushed aluminum finish better.
• 7200RPM 2.5″ drives probably won’t work with just 1 USB port for power. A USB port can provide up to 2.5W (500 mA at 5V); the 7200RPM drives typically specify at least that much power to operate. Coolmax provided a single USB port to power plug cable and a dual USB (one power, one power & communications) to 5-pin cable. I plug in the eSATA cable first, and then the USB cable so the drive always starts up in eSATA mode.
• The Hitachi drive runs a little warm, but not hot. The Acomdata drive doesn’t even get warm.
• The eSATA removal procedure is a little more involved than on USB drives. One approach is to use the hardware device manager (e.g. provide convenient link to devmgmt.msc). Sometimes SATA drives (including internal ones) show up on the Windows “Safely Remove Hardware” icon. If they don’t, sometimes the Hotswap! utility can help (it does not support all SATA controllers).
• Frequently adding/removing eSATA drives may cause problems with software activation schemes. I suspect it may be more of a problem when Windows thinks it’s a permanent (instead of removable) drive.
• The eSATA/SATA standards have their quirks. Many early add-on cards do not support hot swap; probably most early motherboards do not either. SATA is point to point, so if your motherboard comes with only 2 SATA connectors, and you’ve already got two HDD’s, you’re going to have to add an eSATA card. SATA and eSATA connectors are slightly different. The eSATA cable is better than IDE, but not as flexible (or long) as USB, although I found a 6 ft model which seems more flexible. eSATA cables seem to be $10 or more.
• Since one PC needed more SATA ports, I bought a Vantec UGT-ST300 eSATA PCI card for $30. It was the cheapest card the got good reviews and guaranteed being able to hot swap. I haven’t had any problems with it yet.
• On the other computer, I have a bracket that converts a SATA cable (from the motherboard) to an eSATA connector.
• Since eSATA is not common, it’s best if you only use a few computers regularly with the portable HDD. And it’s convenient to pre-stage the cables – have a eSATA cable and USB power cable already plugged in to the computers you use the most (and carry an extra dual USB cable with the drive for other computers).

Note 4/19/2011: right now I’m basically running everything on my laptop.  If I wanted killer portable storage now, I’d look at combining a superfast SSD in a 1.8″ case with a USB 3.0 interface (if such a thing exists).  Maybe a USB 3.0 memory stick would work well, too.  But don’t forget to back up — flash mass storage can fail catastrophically too.

Tony

OK, this isn’t just for VB6, but I needed to call a function in a C DLL from VB6, and VB6 can only call functions that use the stdcall calling convention.

A properly written C DLL is easy to use from VB6, but a C DLL not written with VB6 in mind may require an adapter layer. C++ DLLs are typically very difficult to use from another compiler (or VB), due to incompatibilities in areas such as name mangling. Of course, COM DLLs and .NET assembly DLLs are totally different cases. COM objects are almost always easy to call from VB6. .NET assemblies are like C DLL’s: when properly designed with a COM interop, using them is like using a COM object, but if they’re not designed with COM in mind, then they can’t be used.

If you have the source or header files, you should be able to determine the calling convention – although you may have to know a bit about the compiler, too (so you know what the default convention is, and how each convention is identified).

A second method that sometimes works is to use Microsoft’s dumpbin program. You can get dumpbin for free by downloading and registering Visual C++ 2005 Express Edition (notes on Win32 development here) . By dumping the Export list and looking at the decorated names, you can determine the calling convention. I got the idea for this approach from here.

For a function defined as void foo(int a), its decorated name would be:

• _foo for __cdecl (C calling convention)
• _foo@4 for __stdcall (Windows calling convention)
• @foo@4 for __fastcall

I used dumpbin /EXPORTS path_to_dll from the Visual C++ 2005EE command prompt. However, not all DLLs export the decorated names, even for DLLs created with MS VC++. I’ve seen some that do and some that don’t; it might have to do with how the exported names are declared (e.g. in a DEF file). Non-Microsoft C compilers (MingW, gcc, Borland, etc) might decorate names differently.

By the way, you can use dumpbin /DEPENDENTS path_to_DLL to see what DLLs a DLL requires.

Another way is to use Python and the ctypes module. The ctypes module is included in Python 2.5; it’s an add-on to earlier versions. ctypes can load a DLL as either a Windows (stdcall) DLL using windll.LoadLibrary or a C (cdecl) DLL using cdll.LoadLibrary.

Here’s an example using Python and two functions in the MEI PC/DSP 2.5.09 MEDVC60F.DLL. One function (dsp_init) is stdcall; the other function (find_pci_dsp) is cdecl.

>>> from ctypes import *
>>> stdcall = windll.LoadLibrary(‘C:\\medvc60f.dll’)
>>> cdecl = cdll.LoadLibrary(‘C:\\medvc60f.dll’)
>>> ShortArray20Type = c_short * 20
>>> addr = ShortArray20Type()
>>> irq = ShortArray20Type()
>>> numBoards = c_short(0)
>>> cdecl.find_pci_dsp(byref(numBoards), byref(addr), byref(irq))
0
>>> stdcall.find_pci_dsp(byref(numBoards), byref(addr), byref(irq))
Traceback (most recent call last):
File “<interactive input>”, line 1, in <module>
ValueError: Procedure probably called with too many arguments (12 bytes in excess)
>>> cdecl.dsp_init(0×300)
Traceback (most recent call last):
File “<interactive input>”, line 1, in <module>
ValueError: Procedure called with not enough arguments (4 bytes missing) or wrong calling convention
>>> stdcall.dsp_init(0×300)
2

even if you’re using C# or VB.NET. How? By using Boo’s interpreter to try out ideas, test usage of .NET framework functions, and interact with components (COM, .NET). Then when you’re comfortable, you can write the polished code in the language of your choice.

This is one use for mixed language programming, mentioned by me here, but where the final software might not use both languages.

In fact, any .NET interpreter such as IronPython or IronRuby could be used. And it’s a great technique for other than .NET – I’ve used it a lot with Python on Win32, and you can use it in Java with JRuby, Jython, Groovy, etc. This approach could be very useful for embedded and factory software development.

On .NET IronPython and IronRuby have advantages because they’re official Microsoft languages (and IP has a book coming out – IronPython In Action) and the languages are already in wide use (Win32, *nix, Java). I’m using Boo because it comes with the open source SharpDevelop IDE, and right now it integrates better with .NET (IP does not support attributes well, and it’s not easy to make an IP assembly callable by other .NET languages, etc).

Short example – interactively using DirectoryInfo in Boo (text I typed in bold):

>>> di = DirectoryInfo(“C:\\Download”)
C:\Download
>>> files = di.GetFiles()
(AdbeRdr80_en_US.exe, AdbeRdr80_en_US_Nosso_error.log, Firefox Setup 2.0.0.1.exe, SharpDevelop_2.2.1.2648_Setup.msi, TortoiseSVN-1.4.3.8645-win32-svn-1.4.3.msi)
>>> files = di.GetFiles(‘t*’)
(TortoiseSVN-1.4.3.8645-win32-svn-1.4.3.msi)

Interactively calling a COM object in Boo:

>>> ieType = System.Type.GetTypeFromProgID(‘InternetExplorer.Application’)
System.__ComObject
>>> ie = System.Activator.CreateInstance(ieType)
System.__ComObject
>>> ie.Visible
false
>>> ie.Visible = true
true
>>> ie.Visible
true

So far I haven’t had to use InvokeMember to call COM functions. At least for C# and VB.NET (and probably Boo) it appears you need to use it if you are using late binding. BTW, it is possible to use late binding with COM events, but it is significant extra work.

Tony

I was going through an old stack of JDJ‘s (Java Developer Journals), and saw an interesting commentary on Test Drive Development (TDD). The author said its purpose was to help you write the minimal amount of code necessary to meet the current requirements, and that having a test suite available was a nice side affect. His TDD development sequence is:

1. Write tests that show your requirements, based on the perfect client interface
2. Write just enough code to compile successfully
3. Write just enough so that all the tests pass
4. Refactor

Now apply this idea to the world of factory automation software. Hmm, could be interesting, but it wouldn’t work with PLC’s (rllUnit anyone? I don’t want to even imagine porting xUnit to IEC61131.)

Tony

Here is my quick overview of modern software development trends. It’s somewhat quirky, but software development is a big area, all the way from programming $0.50 PIC micros in toaster ovens to massive web services and supercomputers.

Where do I pick up these trends? From trade magazines (after applying my hype discount factor), web blogs, discussions with other programmers, and what works for me.

• Version Control and Bug Tracking that work. These are essential for good development, but I suspect that most automation developers, at least at small companies, still do not use them.
• Design Patterns – a good way of talking about software architecture.
• Unit testing and Test Driven Development – somewhat hard to apply to automation software, but definitely making waves right now in desktop and web development.
• Refactoring – restructuring and improving existing software, instead of doing a ground-up re-write.
• Agile Development – iterative, customer focused development instead of BDUF (Big Design Up Front).
• Dynamic languages – allow rapid development, better re-use of existing components, run time customizations.
• Mixed language development – use appropriate tool for the job. A desktop example is Adobe Lightroom, written in Lua and C. I do a lot of this (especially when including IEC 61131 as a language)
• Web technologies – interesting stuff, but I think factory software developers should not get caught up in the hype.
• Open Source – some very good products to add to your toolkit.

I plan on discussing many of these techniques in the future, using concrete examples of the tools I use and like. For example, I will use Subversion when discussing version control.

Comments 4/5/2011: still good stuff, but unfortunately I haven’t followed up (limited by lack of time and my fun excursions into PCB and MCAD land).  I need to write more about these topics.

Tony

Open source software is a great resource. If you’re going to use and modify open source code in your machine, then you have to look at licensing issues very carefully. But there is another way to take advantage of open source software – by using open source projects such as Subversion to enhance your development process with no licensing worries.

Even if the majority of open source software can safely be ignored, there are many rock solid, high quality, well documented, Windows-friendly projects. Examples include:

• Subversion for version control with TortoiseSVN client for Windows
• Trac for project management and bug tracking
• Apache web server
• xUnit for unit testing
• Wix for MSI installers (OK, its documentation is a little weak, but it’s still what I use to create Windows installers)
• Programming languages such as Python, IronPython on .NET, and Lua.

What are some of the advantages of these top notch projects?

• High quality. For example, Subversion has a better reputation than Microsoft’s SourceSafe.
• Low cost, both in money (nothing) and time (due to good documentation).
• Ease of deployment – with Subversion, I don’t have to worry about how many users there are. Adding another user costs exactly $0.00, unlike adding a user on a commercial version control system
• No licensing hassles; there are no hardware keys, no software keys, and no chance of any software license audits.
• More stable path – no forced upgrades (upgrade when you’re ready), no worries about the vendor going out of business, no vendor agendas to worry about.
• Often commercial help is available if desired – so far I’ve done fine with the documentation and Google.

Tony