Emerald AI

A Machine Learning platform for collecting, cleaning, and balancing data

Krydus AI Logo

Emerald AI

Emerald AI is a turnkey solution for data collection, cleaning and processing that both accelerates AI development and reduces costs. By combining a large number of openly available data sources and the ability to add private sources for collection and processing vast quantities of data can be prepared. The platform supports quickly creating new object types for search and filters for what should or should not be represented in a particular dataset including attributes like seasons, weather, time of day, geographies, and any other type of attribute the data scientist deems significant.

With Emerald AI the interaction required to build a large scale dataset is reduced to entering a few keywords and example images and from there the process is completely automated. When data is exported, either in a format ready for labeling or directly to training, the machine generated suggestive labels are included. Data can be exported in many standard formats or can be streamed directly for unsupervised training.

Generating a continuous stream of never before seen data solves another challenge facing current large supervised deep learning models, edge cases. The hardest part of building robust models, especially in safety-critical settings, is handling situations that happen uncommonly. Emerald AI can provide a constant stream of never before seen data instead of the currently used method of looping over the same dataset approximately 100 times.

How can enough data quickly and cheaply be obtained to train modern supervised deep learning models?

Over the past decade, Supervised Deep Learning has revolutionized computer vision and consistently set new benchmarks. It has created previously impossible applications. Despite this, the current models are too data-hungry to be useful for many applications and sometimes prevent innovation due to large data requirements to create a proof of concept. It is widely understood that the hardest part of building AI is obtaining enough data and especially how it deals with situations that happen uncommonly, i.e. edge cases. Additionally, as model accuracy increases more accurate data is needed to improve it. This has caused exponential increases in the cost to improve AI systems. Machine Learning engineers and data scientists need tooling to quickly collect, clean and package large amounts of data for modern AI pipelines.

The Solution

Data scientists and machine learning engineers from junior to senior roles all agree that the data collection, cleaning, and management phase is hardest. The solution is a data collection, cleaning, and management platform that can greatly accelerate development by collecting and processing data 100 times faster than traditional solutions.

Vast amounts of data exists freely available on the web ready to accelerate AI learning for many applications it only needs to be leveraged. Clever teams will do so at the cost of months of work writing custom scraping tools for their application before even starting to develop real solutions to business problems. Luckily this can be completely automated using AI enabled scraping with minimal interaction and supervision from data scientists. Further dataset statistics can be generated and datasets can be automatically balanced using this information by actively seeking out the type of data that is required.

Emerald AI provides an automated data wrangling solution, freeing data scientists to focus on developing their models

Benefits

  • Fewer expensive data scientists needed to staff projects
  • Completed projects in a fraction of the time
  • Drastically lower costs / quicker time to value for organizations
  • Scaling by rapid iterations for new use cases

capabilities

  • Custom Machine Learned Classifiers
  • Auto-Generation of Dataset Statistics
  • Auto-Formatting and Cleaning of Datasets
  • Detailed Data Visualizations
  • Quick Onboarding through SaaS
Emerald vs Traditional AI
Collecting/Ingesting

Collecting data using our marketplace or built-in web scraping functionality, allows Data Scientists to rapidly augment their data when there is a shortfall. Ingesting data into the platform is quick and easy from cloud or local sources.

Filtering/Attribution

Custom Creation in as little as 2 minutes, giving Data Scientists precise filters and attributes to apply to their data-sets. Emerald AI provides over 300 initial attributes (and growing). Filters trained on 5-10 images vs 500-1000 for other solutions.

Balancing

Connectors (Data Sources), with the assistance of Filters and Attributes, allows the Data Scientist to quickly and easily see what areas of the data-set need balancing and apply an additive filter for easy adjustments, before training anything.