AI & Machine Learning Data

Developments in AI and machine learning are published across many kinds of sources: company websites, product pages, documentation sites, research portals, technology directories and news sources. A single organization can appear with its products and tools, the models it releases, the research it publishes and the announcements it makes.

Each source presents these in its own format, and the same product or model can be described in several places. Structured collection links organizations, products, models, publications and announcements, and records when each was collected.

Request a Data Collection Project

AI and machine learning data is relevant to organizations that need a structured view of what is publicly displayed about AI, including:

• AI and machine learning teams researching models, tools and technologies
• Researchers working with publication and topic metadata
• Product researchers looking at AI products and their displayed features
• Organizations building structured datasets about the AI landscape

AI and machine learning information has a structure of its own. An organization offers products and tools, releases models in versions, publishes research and makes announcements, and each of these is a different kind of record. Linking them, and keeping them distinct, is what makes a dataset about AI usable.

The data described here is information about AI and machine learning as sources publicly display it. It does not describe collecting datasets for training AI or machine learning models.

AI and machine learning data is easier to use when it is clear what each record represents. The organization links the record types: its products and tools, its models and versions, its research publications and its announcements. AI category and topic can also link them, where a source displays them.

An organization is recorded at business level: its name, its website, its location and its AI category, where displayed. The products and tools it offers link back to it, each with a category, a description and features and, where publicly displayed, pricing, supported use cases and platform information.

The same organization or product can appear on more than one source under different names. When a project draws on more than one source, matching decides, where possible, whether two records describe the same organization or product. The fields available for matching depend on what each source displays.

A model is recorded separately from products and tools. It has a name, a model type and the organization that publishes it, and it can have several versions. Where publicly displayed, a model record also carries its supported capabilities, its context or input information, its release information and its technical specifications.

Sources describe models at different levels of detail, so the same model can carry different fields on different sources.

A research publication is recorded through its metadata: the title, the publication URL, the publication date, the topic, the category and the keywords, together with the abstract or summary as it is displayed. The publishing organization is recorded where displayed.

Authors are recorded as the byline, the names as the source displays them, and author data does not extend beyond the byline. The full text of a publication is not part of this scope.

Announcements and product updates appear on company websites and news sources. They are recorded as metadata: the headline, the article URL, the publisher, the publication date, the company and product concerned, and the displayed summary. Article text is not collected.

Topics, categories and AI categories are the labels sources use to classify these records, and they can link organizations, products, models and publications where a source displays them. Sources classify topics in their own way, so labels from different sources may need to be aligned.

The data types below reflect what AI and machine learning sources commonly display. Which of them apply depends on the project and on what each selected source publicly shows.

AI product and tool data describes AI software, platforms, tools and products as a source displays them. Where publicly displayed, it may include:

• The product name
• The product URL
• The company
• The product category
• The description
• Features
• Pricing, where publicly displayed
• Supported use cases
• Platform information, where displayed
• Other publicly displayed product attributes

Company data describes the organizations behind AI products, at business level. Where publicly displayed, it may include:

• The company name
• The company URL
• The industry
• The AI category
• Products
• Services
• The company description
• The location
• Other publicly displayed business attributes

Model data describes AI and machine learning models as a source displays them. Where publicly displayed, it may include:

• The model name
• The model URL
• The organization
• The model type
• The version
• Supported capabilities
• Context or input information, where publicly displayed
• Release information
• Publicly displayed technical specifications

Publication data describes research as a source displays it. Where publicly displayed, it may include:

• The paper title
• The publication URL
• The author byline, as displayed
• The publication date
• The topic
• The category
• The displayed abstract or summary
• The publishing organization, where displayed
• Research keywords

Publication data is metadata only. Author profiles, contact information and affiliation histories are not collected.

News and announcement data describes AI-related news, announcements and product updates as a source displays them. Where publicly displayed, it may include:

• The headline
• The article URL
• The publisher
• The publication date
• The company
• The product
• The topic
• The category
• The displayed summary

News and announcements are limited to metadata. Article text is not collected.

AI and machine learning data can come from a range of publicly accessible source types, including:

• AI company websites
• Research portals
• Technology directories
• Product pages
• Documentation sites
• News sources
• Other online sources specified for the project

Collected information remains subject to each source’s access conditions and permitted use requirements. Not every source type applies to every project, and each source presents products, models and publications in its own way.

AI and machine learning data projects usually start from a question about products, models, research or organizations. What the data describes is limited to what the selected sources publicly display.

Building a structured view of the AI landscape from selected sources: which organizations offer which products and models, what capabilities and versions they display, and how they are categorized.

Building structured datasets of AI tools, platforms and products, with their categories, features, supported use cases and pricing where publicly displayed.

Working with research publication metadata: titles, publication dates, topics, categories and keywords, with the author byline and the abstract or summary as they are displayed.

Building structured datasets of information about AI from selected sources: products, models, publications and announcements linked to the organizations behind them, with each record carrying its collection timestamp.

AI and machine learning data projects are defined around the sources, the organizations in scope and the record that each row should represent. The main points to settle are:

• Source websites
• Organizations in scope, limited to business-level information
• Products and tools in scope
• Models and versions in scope
• Research topics and publication scope
• Announcement and news metadata scope
• AI categories and topics
• Required fields
• Pricing scope, where publicly displayed
• Record level: organization, product, model, publication or announcement
• Matching requirements
• Collection dates and timestamps
• One-time or recurring collection

The scope is limited to publicly displayed, business-level information about AI and machine learning. Author information is limited to the byline as displayed. Individuals, personal contact information, author profiles and affiliation histories are outside the scope of this data.

A one-time collection captures selected sources at a single point and can support:

• AI technology research
• AI product research
• AI research publication research
• AI dataset creation

Recurring collection repeats the same sources and fields as timestamped snapshots, so that new and updated products, models, releases, publications and announcements can be compared over time. How often data is collected depends on the project requirements. Each snapshot reflects what a source displayed when it was collected, and coverage depends on the selected sources.

A clear requirement makes it easier to choose the right sources and record structure. A project can be defined by the sources, the organizations, the products, models and publications in scope, the fields, the record level and the collection schedule. Once these are defined, we can review the project scope and confirm an approach.

Request a Data Collection Project