Tag Archives: Datamyne

Datamyne, Part 4: A Trade Analyst’s Dream Machine & WTD Editorial Note

Last year we ran several articles on the TI Provider Datamyne, Lisa Wallerstein the VP Product Development /Marketing and the announcement of their new interface, Datamyne 2.0.  In this last week’s series of articles, I focused on the specifics of their TI product following an in-depth demo and review.

I must admit that recently, while making my rounds with each top-tier TI provider and getting a deeper look at their products, my viewpoint is changing.

What I am discovering is that there is no one best product and TI provider out there.  Each has its own particular approach, technologies, interface, data sources, value added services, and pricing model that make it the best solution for certain niche or target markets.

After reviewing Datamyne’s interface, I can confidently state that it represents perhaps the best analytical user interface by which to “slice and dice” the U.S. Customs data that I have seen thus far.  It’s a trade analyst’s dream machine.

If you’re looking for a robust TI product that is straightforward and relatively easy to use by which to perform heavy-duty analytics on the U.S. Customs Data, you definitely need to check out Datamyne.

WTD Editorial Note: I’d also like to address an important issue that has come up on several occasions.  I have been accused by some of being overly biased toward PIERS.  In a number of articles and in handfuls of instances it has been noted that I have stated that PIERS products are the best.  By and large, I think the accusation has been well founded.

To be honest, one of the reasons for my bias is that the TI products we developed at CenTradeX (IMHO) were vastly superior (in aspects of innovation, UI access, data integration, graphic delivery and performance) in comparison to any other TI product. PIERS acquired these assets last year.  Furthermore, the team of superb technologists I worked closely with for many years also migrated to PIERS.  Therefore my high esteem has been transferred or credited to PIERS posthumously, so to speak.

Secondly though, having dealt closely with the new management team at UBM Global Trade /PIERS for many months during the due diligence process and afterwards (being somewhat privy to their “thought engineering” and witnessing the changes they are implementing) I have gained tremendous respect for them.  Further, they have several proprietary databases that are “untouchable” by competitors.  They have roots in history and experience that go back over 100 years.  They have worked with Customs data along with the nuances of refinement and standardization for many decades.

So, with regards to my previously perceived biases, I hereby recant my previous position that there is ONE best company and ONE consummate Trade Intelligence platform.  Vive la différence!

Datamyne, Part 3: Micro to Macro and Back in 60 Seconds or Less

One of the coolest things about Datamyne’s data mining interface is the drag and drop feature for adding or subtracting fields of data to the display and export processes.  Similar to creating and viewing Pivot tables in Excel, the UI provides users the ability to dimensionalize their perspective of the data and flip it one way then another.  Again, perfect for analysts who require more complex manipulation of the data.

My personal preference when working with or presenting trade data is first of all to get a big picture view, then logically drill down to finer detail and greater specificity.  This ability is absent in most TI products on the market.  Datamyne’s interface provides that functionality superbly.  It allows you to get both the macro view and the micro view as well.

One of the biggest difficulties I labored with in developing trade intelligence platforms is the tension between making data easy to access and to understand on the one side and providing the detail and specificity on the other.  Although Datamyne’s data mining utility is well crafted, it is still rather complex and fairly intimidating at first glance.  It’s about as easy as something complex can be.  Heck, all the buttons and control options on most video games confuse me.

Another important feature is that once users create and perfect their queries, they can save and retrieve them for future use.  In addition, they can set-up alerts that will automatically email them when certain criteria are triggered, such as when new shipments occur or new suppliers or competitors for their designated product enter the scene.

Not surprisingly, Datamyne’s customers are larger companies with complex supply chains who need to perform custom analyses.  To accommodate this market niche, Datamyne provides more extensive customer support and training.  They are the full service solution in comparison with many of the self-service e-commerce type conveyers of U.S. Customs data that have recently proliferated the marketplace.

Other competitive strengths that were mentioned to me include:

  • The best overall value with prices starting as low as $199 per month for access to the raw unfiltered, non standardized U.S. Customs manifest data.
  • Latin American Coverage:  Datamyne has its roots, data center and most employees south of the border (Uruguay). They offer transactional data for many South American and Central American countries.
  • Statistical data gathered and disseminated for almost 50 countries, updated monthly.
  • Versatile and (relatively) easy to use user interface.

All in all, it represents a very good data mining utility for trade analysts to employ to slice and dice and serve up U.S. Customs data.

Datamyne Data Mining Utility provides capacity to quickly get the big picture as well as drill down on a specific Bill of Lading

Datamyne’s way too cool Drag and Drop feature that allows trade analysts to create pivot table like views of the data

Datamyne, Part 2: Queries: the Tiny SQL Fairies that Fetch Data

The front end or UI (user interface) that sits atop of Datamyne data has the look and feel of an analytical tool.  Even the naming conventions bear witness to their particular approach and designated audience.  All three initial windows into the soul of the machine have the label “queries” attached.

Users begin their search by either engaging with “Rankings & Queries”, “My Saved Queries” or “My Most Recent Queries”.  Obviously, first timers’ starting point is the first item.  It all begins fairly simply by selecting a country (market), database (transactional, statistical) and a year. Then, you choose between “rankings” and “queries”.  It’s all pretty straight forward thus far.

If you select “rankings”, you’ll get back an overview of the data arranged by commodity or geography.  If you select “queries” you’ve just bought the “E Ticket” to Data Disneyland or perhaps more like entering a geek construct of Data Lego Land.

The individual building “blocks” are composed of chunks of U.S. Customs manifest data –foreign shipper, U.S. importer, addresses, TEUs, commodity, product description, ports, weight, etc — several dozen fields of information for each individual bill of lading (shipment) multiplied by millions of BOLs each year.  That’s a lot of blocks.

Queries are formal inquiries constructed in SQL programming language posed by users in order to properly extract, filter and display these data blocks in ways that will solve a problem or answer a question.  Good questions get good answers.  Datamyne’s UI allows users to easily engage in the process of creating, editing and revising their query.

Proper question or query formation is more an art than a science as anyone who has tried to find anything via Google or another search engine can attest.  You have to refine and re-refine your search terms in order to filter out unwanted junk and get to the treasures buried in the dirt.  That’s the mining of data mining.

One of the excellent features about Datamyne’s data mining utility is that all the manipulation and revision happens on one page.  You don’t have to jump from here to there or follow links to a different page or start all over again if you want to alter your search criteria.

I was impressed by the relative speed by which the query results were returned.  Perhaps not lightning (instantaneous –POW) speed but certainly thunderous (one thousand one, one thousand two, one thousand three) speed.  That’s a lot of data to sift through, arrange and display.

DataMyne Home Screen. A Query Smorgasbord. Create a New Query. Fetch Saved Queries. Retrieve Recent Queries.

After Sending out their Query Fairy, Users can dynamically interact with it to refine search results

Datamyne, Part 1: Data Standardization and Product Code Attribution

As I have said many times, Trade Intelligence is composed of many facets: data, technologies, application and people.  It’s people that really boost the IQ.  Notwithstanding, on some level it starts and ends with the data.  If you’ve got bad or ugly data to start with… whatever you build ain’t gonna look pretty.  No decision will be well founded.

It is reported that Datamyne’s standardization process of the U.S. Customs data results in linking 85% of the manifest records (that are not suppressed or invisible) to U.S. consignees (importers).  Believe me, that’s no simple task.

As part of the deal Datamyne can filter out NVOCCs and logistics folks. There is nothing more irritating when you’re doing a search through Customs records, looking for importers of a particular product let’s say, than to have your results display 10 iterations of the same importer along with “junk” or fake consignees, which are obviously not importers.

Future iterations of their product promise to link expanded D&B company information to U.S. importers, further enhancing the value.  They believe they will be able to match about 60% of the non-suppressed records to the D&B database.  It’s all about “connecting the dots”!

U.S. Customs data is fetched daily via FTP.  Datamyne processes and makes a “raw” version (without standardization of names or product coding) available online within 72 hours.  They’re intending to compress that time down to a 24-hour turn-around very soon.  Their standardized, coded version is updated twice per month around three weeks in arrears.

More about data.  Datamyne also attributes a Harmonized Code product classification to over 80% of the manifest documents.  Although usually only to the two or four digit hierarchy, this is a fairly remarkable and difficult undertaking.  With the exception of PIERS, no other TI provider does it.  Although they’ve yet to fully exploit the many benefits of these linkages and the many profound synergies that can be created between statistical, and company data sources, they are clearly laying a firm foundation.

Although cloistered in separate silos, Datamyne maintains statistical data (updated monthly) on almost 50 countries within Europe, Asia and Latin America.

Furthermore, they maintain transactional manifest shipping data (similar to U.S. Customs data) on many Central and South American countries.  Unfortunately for Anglos, this data is only available in Spanish.  However, the availability and expertise with Latin American data remains one of their unique competitive strengths.  Datamyne’s origins, data center and most employees are rooted in Uruguay.

Datamyne has a slick “Drag and Drop” feature that allows users to chose which data elements they want to search on, display and export.

Recent Interesting Blogs from Datamyne’s Mine of Data and Posts; PoL v#3.5

Datamyne leads the pack of TI providers in the use of social media. They publish over 200 articles a year on various aspects of world trade data and application. Since we will have featured a couple of articles about them this week, it seems fitting to fetch a few yummy feeds from their blog kitchen-like we do for our weekly Pick of the Litter series.  In this case perhaps better entitled Best of Breed.  Anyway, here are some samples published within the last 90 days:

Topping the list must be the article heralding the release of Datamyne’s new interface cleverly named Datamyne 2.0. Also, available for viewing is a video demo of their new product.

Since they use frozen tilapia as the chosen product (from tens of thousands of possible candidates) to demonstrate the power of their new interface, their recent blog post “Cluck-cluck Cutlet of the Sea” about tilapia is appropriate. Wow, Americans consume 1/2 billion pounds of Tilapia per year. For more fishy facts, check out this post.

From fish to fruit, this post indicates that the U.S. buys 37% of all the fruit exported by Chile with neigh to half a million metric tons of table grapes consumed annually. My daughter loves grapes. Every week, we buy several pounds for her alone. We like to do our part in supporting the global economic growth of our southern cousins. Top Pick: Chilean Fresh Grapes

This recent blog post on Selling Med Equipment in Brazil highlights Datamyne’s capacity to look into the trading patterns of other markets particularly those of Latin America. With the exception of PIERS, no other TI provider (to my knowledge) collects and disseminates transactional data of this sort. This is great for U.S. Exporters who are looking for markets to the south.

Who’s Open for Business? World Bank’s annual report is a treasure trove of free information. This is an interesting post that references a report on 183 countries that ranks the overall ease in doing business overseas. For instance, it takes an average of 7 documents, 17 days and $1,730 per container to comply with the governmental procedures required to export a shipment from the U.S. to Brazil.

Americas Barometer 2010– Taking the measure of democratic values and the impact of the downturn, do our Latin American Neighbors believe Free Trade Works? This blog article, featuring a attractive and descriptive graphic, charts the percentage of people (organized by country) that think “free trade” has helped their economy. What do you think? Does it?

TI Professional Lisa Wallerstein, V.P. Product Development & Marketing for Datamyne

I’ve known Lisa Wallerstein for 7 or 8 years.  I first met Lisa when she was employed by PIERS in a similar position as the one she now holds with Datamyne- Vice President of Product Development and Marketing.

Back in the day, she was one of the voices calling for product innovation and development.  Trying to spark change within an aging behemoth corporation can be challenging.  I think she was perhaps, at the time, the most valuable asset PIERS possessed.

She is, and has been, on a continued quest to “connect the dots” of International Trade Data in order to provide enhanced dimension, transparency and practical utilization to clients.  One of her most impactful projects, while she was with PIERS, was to link D&B company data to the daily shipping manifests.  By so doing, users could more accurately identify the actual trading partners (buyers and sellers) beneath the numbers.  Another project she shepherded was weaning clients off of PIERS’ monthly CD distribution of the data in favor of online access.

Ms. Wallerstein represents an example of a firm conviction I’ve formed from my 10+ years in the industry; that people are THE invaluable, irreplaceable facet of  “intelligence” behind Trade Intelligence.  The value inherent in TI products can never be reduced to simply technology and data.

Many times people like Lisa are plagued, in whatever corporate context they find themselves, with an uphill battle like Sisyphus… struggling to convince management of the merits of their new ideas and the need for innovation.  However, when the “powers that be” finally give “buy in” and support product development with the needed resources, revolutionary results can be achieved.

In her short 18-month tenure at Datamyne (following a short stint with McGraw Hill), Lisa–along with Brendan R. McCahill (former CEO of PIERS, now CEO at Datamyne)- has been able to work her magic again with the recent release of a new TI product, one she promises, “will make searching trade data easy”.

The international trade marketplace has changed drastically over the last several years.  There are many new competitors in the Trade Intelligence field as well as increasing global economic pressures.  The growth and survival of PIERS, Datamyne and other TI mainstays might well depend upon swift and accurate product innovations and other “value added” services that are developed in response to changing client requirements.

Datamyne Launches Their New Trade Intelligence User Interface- Datamyne 2.0

Datamyne, Uruguay based TI Provider, with U.S. headquarters in Miami, recently announced the launch of their new interface under the name Datamyne 2.0. My personal opinion is that products should each carry their own identifier, even if preceded by the company title such as Ford Mustang, Chevy Camaro, and Jeep Renegade.  A product name should give clients a taste of the character and attributes of the product they own.  Fortunately, a summary review of Datamyne’s new interface proves more innovative than the name implies. So for my own amusement, in this article I will refer to Datamyne 2.0 as D2.

D2 comes in two sizes: one that looks at markets (particular countries’ import/export patterns) using aggregated statistical data. That’s the leaner D2 Jr.  And the more hefty hunk of an interface, handsome D2 senior, that provides subscribers drill down capability to fetch and report on individual BOLs – Bills of Lading.

Lisa Wallerstein serves as Datamyne’s Vice President of Product Development and Marketing. She, along with a talented group of Latin American technologists located in Montevideo, developed Datamyne 2.0 over the last 6 months in response to, and in collaboration with, client’s feedback and suggestions.  She reported that Datamyne 2.0 “was built around the workflow of our customers” in an attempt to make “searching trade data easy”.

Glamour shot featuring D2 with some lovely graphics

D2 is up against several other contenders for TI market dominance, including Zepol, Import Genius, and Panjiva as well as a host of wanna be bottom feeders.  Of course their biggest challenge comes from the dominant species, the 500 pound gorilla of TI providers: UBM Global /PIERS.  (BTW, PIERS names their products: “Prospects”, “TI”, “StatsPlus”, “MyPiers”, “iPiers”, “Trade Finance”.  Granted,  “I”, “My” or “TI” are not the most inspired of naming conventions either.)

D2 has some cool features. The logic behind the interface seems to be built with the end user in mind.  The process follows a well marked path that an average dummy (like me) might take to search, sift, organize and report on the information they’re looking for.  The interface appears well laid out; trimmed in eye pleasing green and blue with ample white space and easy to follow navigational elements.  Another neat feature is that a user can “save” his or her query (question) as an “alert” which will consequently send out an email notification when new data arrives. User reports can be directly exported anywhere along the process to Excel along with graphic representations of the data.

Congratulations to Datamyne on their new product launch and their contribution to the expanding and exciting world of Trade Intelligence.

Finally, check out this 4 minute YouTube Datamyne video featuring the highly acclaimed and handsome hunk of an interface, D2. By the way, WorldTradeDaily.com has a substantial library of TI videos.  You can access them via our VodPod widget conveniently located in the right navigational area on this website.

WTD features: U.S. Customs Data, Foreign Sourcing and Datamyne 2.0 Launch

WorldTradeDaily.com reports daily on the World of Trade Intelligence: the information suppliers, their products, types and kinds of trade data, business applications, and most importantly, the real intelligence behind trade intelligence – people.  We’ll start this week by looking at a new product.

Several weeks ago, Datamyne announced the launch of their new interface, Datamyne 2.0. According to Lisa Wallerstein, Vice President of Marketing and Product Development, their new Trade Intelligence product, “makes searching trade data easy”.

Datamyne, founded in Uruguay in 1992, established its U.S. headquarters in Miami in 2005 with financing provided by venture capital group Nassau Point Investors.  A veteran trade intelligence provider with 120 employees, they offer access to trade information in 48 countries.

In their basic entry level package, subscribers search and report on aggregated statistical trade flow data that highlights the markets that are growing or declining as well as specific volumes and values on 20,000 plus commodities.

Premium users get additional features.   Called Datamyne “Cargo”, subscribers can drill down to the specific details on individual shipments.  By sifting through shippers Bills of Lading (BOL), U.S. manufacturers thus identify and vet potential foreign suppliers of raw materials or components (Foreign Sourcing).  Further, they can see which sources their competitors buy from (Competitive Intelligence).

Datamyne’s new interface was developed over a six month period following extensive interaction and collaboration with their existing clients.  As Ms. Wallerstein further reports, “Datamyne 2.0 was built around the work flows of our customers… how they like to access and use the data.”

The U.S. imports far more goods than it exports (trade balance).  Along with this unchanging reality, American manufacturing has been on the decline for decades.  Instead, we turn to more economical foreign sources for goods (and services). With these trends in mind, Datamyne’s mainstream clientele are using the interface to assist in foreign sourcing.

This week at WorldTradeDaily.com, we will explore:

  • Trade data and how it is used to help U.S. manufacturers and distributors find new sources for their raw materials and product components.
  • U.S. Customs (AMS) waterborne shipping manifest (BOL) import data – what it can tell you and what it can’t.
  • A deeper look at Datamyne’s new Trade Intelligence product, Datamyne 2.0.
  • Lisa Wallerstein and her impact on the World of Trade Intelligence.
  • Continuation of our Pick of the Litter series, fetching our favorite trade related blog posts and articles from the web.
  • Part 2 of WTD Back Story reflecting on the early days of CenTradeX, which became a major innovator in the field.