Zepol, Part 1: Fast, Faster, Fastest… Freakin’ Crazy Fast Search Engine

In this first of four articles on Zepol, I will dispense of the usual company background, forego superfluous narrative that indirectly ties my extensive knowledge and experience in Trade Intelligence to the subject as well as and any other attempt to subtlety impress and go straight to today’s main point.   Zepol’s TI interface is fast.  It is really, really fast.  It is incredibly, spectacularly, spine tingly fast.

As is my M.O., during the product demo provided by Zepol President Paul Rasmussen, we dispensed with the canned presentation and went off the beaten path to look up Scooters with foreign suppliers and corresponding U.S. importers of under 50cc moped style motorcycles.  The default search scope is set to hunting within the last 30 days of Manifest records.  In a second or two we got the results.  O.K., not too bad, pretty fast.  However, I wanted to see how the search engine would perform when really put to the test.

I asked Paul to extend the search from the very first manifest record, back in January 2003 to the very last record in September 2011… that’s around 100,000,000 bills of lading!  Keep in mind we’re searching through multiple textual fields (“Products” and “Marks & Numbers”) for each BOL using text terms: “Moped”, “scooter” and “50cc”.  We’re not talking about numerical fields with a singular numerical criterion.

BAM!  3-4 SECONDS later we’ve privy to over 6,000 shipments – with corresponding detail if needed – on the representative international trade transactions involving imported mopeds.  WOW.

I refrained from inquiring about the specific alchemy that created the magic.  What combination of server arrays, multi-core processors, RAM, query optimization and full text indexing was employed to do this? Paul credits co-founder Jeff Wilson and his tech team.

A couple of illustrations come to mind that may communicate this amazing data feat better.

I went with my eldest son to the Brickyard 400.  I remember pressing our faces next to the racetrack fence as scores of NASCAR speedsters zipped by at 200 mph.  The visceral memory of sheer speed and power is unforgettable.

In the final scene of the movie Secretariat, the remarkable story of the 1973 triple crown winning race horse, said underdog (rather under-horse) soundly beats the favored “alpha dog” (alpha-horse) by an astounding 31 lengths (still holds the record for speed and margin of victory).

Other notable examples could include Bruce Lee’s unmatched speed performing various martial arts maneuvers, Superman’s counter clockwise planetary orbits to save Lois Lane, and the Enterprise when it hits the warp speed button.

Now maybe you get the picture of just how fast Zepol’s search engine goes.  Zoom, Zoom!