In this article, we give an overview of our Bids & RFPs dataset.
Dataset Description
GovSpend endeavors to be the one-stop-shop for Bids and RFPs from all public agencies in the US. Our aim is to publish Bids to the platform, with all associated details and attachments, within 24-48 hours of an Agency’s public disclosure.
Data Sources/Collection Methods
Bids & RFPs are sourced through several means. The most common method is by using automated technology to harvest the information directly from Agencies as well as from government transparency sites. We also have partnerships with third party aggregators and other services to help bolster our collection. Coverage grows consistently over time as new sources are found. Once identified - and of course depending on priority - our teams can turn around new sources in a day or so.
Additionally there has been some preliminary success in utilizing FOIA requests for getting supplemental bid information such as scoring metrics used by the agency, details of the winning bids, and more.
Refresh Rates/Timeliness
Data scrapers run daily, searching for new proposals posted, downloading associated information, normalizing it, and publishing new records. Bids and RFPs are generally published on GovSpend within 24-48 hours of the agency making it available. For records received from partners, we generally see a larger delay. Continued efforts are being made to improve the timeliness of our postings.
Data Coverage/Comprehensiveness
Overall Coverage: All 50 states in the US, parts of Canada.
- Open Bids: 100-120K Bids
- Federal Bids: 10,000,000+
- Expired Bids: 17,000,000+
- State & Local: 6,700,000+
- Bid Sources: 1,400+
- Bid Scrapers Daily: 9,500+
Frequently Asked Questions
Why don’t we have this Bid? Missing Bid Attachments? Incorrect Bid Details?
Our goal is to obtain all publicly available bids with any and all details and attachments included and published on our platform.
A bid is typically missing from our platform for one of three reasons, either it is hosted by a website that we are unable to utilize due to their terms of service, the agency website has changed requiring us to update our scraper, or it is a new source for us requiring a scraper to be built.
When details are incomplete or attachments are missing that could be due to a change to the agency website requiring us to update our scraper, the posting on the agency website could be inaccurate when we capture it, or it may be erroneously provided to us via a third party source.
The Bid is added close to/after the due date - how can we get this on the platform sooner?
Our intention is to have the Bids readily available on GovSpend as close to as when the Agency makes them available as possible. Our expectation is that through our process, we should have the Bids available within 24-48 hours of the Agency making them available.
Typically, when a bid is delayed it is provided to us via third party source or when an Agency has unexpectedly updated their website requiring us to update our sources. The goal is to always have them available as soon as possible, regardless of the source.