Market Intelligence Data

Information about a market is spread across many kinds of sources: company websites, directories, marketplaces, industry publications, news sites and review sites. Each shows a different part of the market, such as the companies in it, what they offer, what prices they display and how they are rated.

A market dataset brings these parts together around a defined scope, such as an industry, a category or a market segment, and records when each part was collected. Structured collection links company, product, price and announcement information to the same market scope. Market intelligence data focuses on timestamped snapshots of a defined market, while competitor data intelligence focuses more specifically on tracking and analyzing individual competitors.

Request a Data Collection Project

Market intelligence data is relevant to organizations that need a structured view of a defined market, including:

• Market researchers studying industries, categories and segments
• Product and marketing researchers looking at what is offered in a market
• Analysts building structured market datasets
• Organizations combining company, product and price information from several sources

A market dataset is different from a dataset about one company or one product. It is defined by a market scope: an industry, a category or a market segment. Companies, products and services, displayed prices, aggregate ratings and announcements are then collected for that scope and linked to it. Each collection carries a timestamp, so the dataset describes the market as the sources displayed it at a point in time.

Market data is easier to use when it is clear what anchors it. The market scope is the anchor, and companies, products, prices, ratings and announcements attach to it. Deciding what the scope is, and what each record represents, is one of the first choices in a market data project.

The market scope defines what belongs in a dataset. It can be an industry, a sub-industry, a category or a market segment, with locations where a source displays them. Public industry information, where a source displays it, is recorded against the scope.

Sources classify industries, categories and segments in their own way, so classifications from different sources may need to be aligned when data comes from more than one source.

Within a market scope, companies are recorded at business level: the company name, its website, its industry and business category, and its locations. The products and services a company offers link back to it. Research centered on the company itself, rather than on a market scope, is covered by B2B market research data.

The same company or product can appear on more than one source under different names. When a project draws on more than one source, matching decides, where possible, whether two records describe the same company or product. The fields available for matching depend on what each source displays.

Products carry the attributes a source displays: category, brand or company, features and specifications. Where displayed, they also carry a current price, an original price, discount information and availability.

A displayed price is a point-in-time observation. It is what the source showed when the data was collected, and it is recorded with the collection timestamp.

Each collection is a timestamped snapshot of what the selected sources displayed for the market scope. When snapshots from different dates are set side by side, observable changes become visible, such as new products, updated products, price changes, changed categories, new listings and announcements.

Announcements and news are recorded as metadata only: the headline, article URL, publisher, publication date and displayed summary.

The data types below reflect what market sources commonly display. Which of them apply depends on the project and on what each selected source publicly shows.

Company data is limited to business-level information. Where publicly displayed, it may include:

• The company name
• The company URL
• The industry
• The business category
• Products
• Services
• Locations
• The company description
• Other publicly displayed business attributes

Product, service and pricing data describes what companies offer as a source displays it. Where publicly displayed, it may include:

• The product or service name
• The product URL
• The category
• The brand or company
• The current price
• The original price, where displayed
• Discount information
• Features
• Specifications
• Availability, where displayed

Industry and category data describes how a market is classified. Where publicly displayed, it may include:

• The industry
• The sub-industry
• The category
• Companies
• Products
• Services
• Locations
• The market segment
• Publicly displayed industry information

Market activity data describes what appears in a market between snapshots. Where publicly displayed, it may include:

• New products
• Updated products
• Price changes
• New company information
• Changed categories
• New listings
• Selected announcements

Announcements and news are limited to metadata: the headline, article URL, publisher, publication date and displayed summary. Article text is not collected.

Ratings data describes how a source rates a product or company. Where publicly displayed, it may include:

• The rating
• The review count
• The product or company
• Rating categories, where available

Ratings are limited to aggregate figures: the rating and the review count.

Market data can come from a range of publicly accessible source types, including:

• Company websites
• Industry publications
• Directories
• Marketplaces
• News sites
• Review sites, for aggregate ratings only
• Other online sources specified for the project

Collected information remains subject to each source’s access conditions and permitted use requirements. Not every source type applies to every project, and each source presents companies, products and categories in its own way.

Market data projects usually start from a question about a defined market scope. What the data describes is limited to what the selected sources publicly display.

Building a structured view of a market: which companies operate in an industry, category or segment, what products and services they offer, and where they are located, as the selected sources display them.

Comparing the products, features, specifications and displayed prices that selected sources show within a category, with availability where displayed. Prices are recorded as displayed at the time of collection.

Looking at the observable changes that appear between timestamped snapshots of a market: new products and listings, price changes, changed categories and announcements, as the sources displayed them.

Building structured market datasets that combine company, product, price, aggregate rating and announcement information around a defined scope, with each record carrying its collection timestamp.

Market data projects are defined around the market scope, the sources and the record that each row should represent. The main points to settle are:

• Market scope: industry, category or segment
• Source websites
• Companies in scope, limited to business-level information
• Products and services in scope
• Locations
• Required fields
• Price basis
• Availability handling, where displayed
• Announcement and news metadata scope
• Aggregate ratings scope
• Record level: market scope, company, product or announcement
• Matching requirements
• Collection dates and timestamps
• One-time or recurring collection

The scope is limited to business-level information that sources publicly display. Individuals, personal contact information and review text are outside the scope of this data.

The data consists of datasets built from publicly displayed information. It does not include forecasts or recommendations.

A one-time collection captures a market scope from selected sources at a single point and can support:

• Industry and market research
• Product and pricing research
• Market landscape research
• Market dataset creation

Recurring collection repeats the same market scope, sources and fields as timestamped snapshots, so that observable changes such as new products, price changes, changed categories and new listings can be compared over time. How often data is collected depends on the project requirements. Each snapshot reflects what a source displayed when it was collected, and coverage depends on the selected sources.

A clear requirement makes it easier to choose the right sources and record structure. A project can be defined by the market scope, the sources, the companies and products in scope, the fields, the record level and the collection schedule. Once these are defined, we can review the project scope and confirm an approach.

Request a Data Collection Project