Web Data Feeds

A web data feed is a recurring delivery model. Information collected from defined sources is refreshed and delivered again according to a schedule set for the project, rather than handed over once as a single export.

A one-time collection produces a dataset once. This delivery model is built around repeated collection and repeated delivery, so the same structure comes back on schedule instead of being requested again from scratch.

Request a Data Collection Project

Every feed starts the same way a one-time project would: sources are defined, and the fields needed for the dataset are agreed with the client.

What makes it a feed is what happens after that first delivery. Instead of stopping once, the same sources are collected again and the structured dataset is delivered on the schedule established for the project.

• Sources defined for the project
• Fields selected for the dataset
• Structured dataset built from those fields
• Refresh schedule agreed for the project
• Delivery repeated on that schedule

A feed can carry a range of structured information, provided the sources and fields are defined in advance. Common examples include:

• Product or catalog records
• Pricing and availability data
• Business or company records
• Listings such as property, job, or marketplace data
• Article or content metadata

The fields available in any of these depend on what each source publishes and what the project defines as in scope.

How often a feed repeats depends on the requirement behind it. Some projects need daily updates, others weekly, and others fit a different cadence entirely.

There is no single frequency built into the service. Each feed follows the frequency defined for the project.

A feed is only useful if each delivery can be compared with the last one. That depends on keeping the same fields, naming, and structure in place across every refresh.

Comparable deliveries depend on a structure that doesn’t change from one refresh to the next. The output format agreed at the start of the project is what every scheduled delivery follows.

On each scheduled cycle, the defined sources are collected again. Whatever is found is reflected in the updated dataset, and the result is delivered according to the recurring schedule set for the project.

This is a collection-and-delivery cycle rather than a monitoring service. The feed does not track or report on what changed between refreshes; it delivers the current structured dataset each time.

A single dataset is collected once and delivered once. This fits a defined research, database, or analysis need that does not call for ongoing updates.

The same sources and fields are collected again on the schedule defined for the project, and each delivery follows the same structure as the one before it.

The stages below outline how a recurring feed moves from defined sources through to delivery, with the same cycle repeating on schedule.

A recurring feed fits situations where the same information needs to be checked and delivered again over time, rather than collected once and set aside. Common examples include:

• Selected information that needs to be reviewed on a regular basis
• A dataset that needs to keep the same structure and fields across every delivery
• A workflow, database, or application that expects updated data to arrive without manual collection each time

Where a requirement only calls for a single dataset, a one-time collection project is the more direct fit.

A feed is built around what the project defines. Before scoping the work, it helps to have answers for:

• Sources to be collected
• Required fields for the dataset
• Scope of the dataset
• Refresh frequency needed
• Output format required
• Delivery requirements for the project

Once these are defined, we can review the requirement and confirm an approach for the feed.

Request a Data Collection Project