A revolution in crystallography

The UMR Crystal Harvesting Workcell is designed to perform the last manual tasks associated with the important field of protein crystallography. This workcell is comprised of an advanced articulating arm robot that is linked with an adaptive vision system and equipped with a library of application-specific tools. The UMR accepts incoming microtiter plates, identifies viable crystals, applies chemical agents, captures target crystals and cryo-cools the crystals in liquid nitrogen, all with minimal human intervention.

Automation is playing a progressively more important role in the science of crystallography. The successful development of a tele-operated Universal Micromanipulation Robot (UMR) provided, for the first time, a means of robotically capturing protein crystals. However, this current system still relies on an operator to direct the capture. New research has paved the way for a fully autonomous version of the UMR. Advanced image analysis software is used to locate a crystal of interest and to measure its position relative to a harvesting tool. This same software then guides the capture of the crystal. Initial implementation of this strategy has successfully directed basic capture routines. More advanced capture routines incorporating crystal depth determination and active tracking are in development. Additional enhancements to the UMR include improved microtiter plate alignment, automated tape removal and high precision fluid handling. These subsystems combine to create an advanced version of the UMR with significantly expanded capabilities and value to the macromolecular crystallography community.

Robotic System

Prototype System

The UMR is predicated upon an anthropomorphic Staubli RX60 6‐axis robot, with access to a library of task‐specific end effectors providing it maximum operational flexibility. The ability of the system to manipulate microtiter plates, cut sealing tape, inspect incubation wells, harvest crystals and cryo‐cool crystals under operator direction has been conclusively established. Additionally Square One has established that this system can use simple harvesting algorithms to autonomously harvest crystals. The Staubli robot is connected to a LabVIEW interface controlling the robot and the peripheral hardware. This hardware includes the X Y stage, the punching station, the camera (including focus mechanism), and the fluid dispensing system.


Harvesting takes full advantage of the capabilities of the UMR system. Machine vision is used to locate the crystals, which are subsequently quantified to determine suitability for harvesting. The derived data ─location, size, and shape factor─ are stored and communicated to the system allowing it to select the appropriate harvesting tool. The arm subsequently acquires the tool and locates it in a nominal location, confirmed and adjusted by machine vision. The punch station next opens the well. A predefined harvesting algorithm automatically acquires the crystal with the selected tool. The system can optionally transfer the crystal into a well containing cryoprotectant previously dispensed by the fluid handling system before flash‐cooling (hyperquenching) the crystal. Due to the speed of the system, multiple crystals can be harvested from the same well.

Machine Vision

machine vision and the automatic detection of crystals

In the baseline system, a digital video camera captures a magnified video image of the well of interest at 30 frames per second. The images are transferred into image analysis software, which uses a series of tools to locate and evaluate objects in the well. The result of this analysis are X and Y locations and an area and shape factor for each crystal identified. The video is also displayed on the user interface to allow for tele‐ harvesting if that option is chosen. Several options for depth perception are being pursued, including confocal reflectometry and edge contrast variation (depth scanning). Optical coherence tomography was also considered and discarded due to size and cost constraints. More advanced vision algorithms are being researched and evaluated. These algorithms (e.g., Canny operators, Hough transforms, and correlation‐ based template matching) will be used to dynamically track crystal motion in real time.

Micro Manipulation

The tele‐operated UMR has successfully harvested crystals as small as 10 microns in diameter. However, the addition of an ultra‐high resolution end effector expands the threshold of manipulation by an order of magnitude. A piezo‐electric actuation system developed in partnership with the University of Wyoming will be incorporated in an end effector with 100 nanometer resolution. The addition of this nested manipulator to the next generation UMR paves the way for harvesting extremely small crystals, for harvesting from micro‐fluidic systems, and for in‐situ diffraction experiments.