The setup is simple. Collect the competitor's job listings once a week with a scheduled run, store every listing under its job id, and compare this week's set with last week's set. Listings with an id you have not seen before are new openings. Ids that were present last week and are gone this week are closed or filled. Everything else is still open. That weekly diff is the whole product. Nothing else in this article is required, the rest is about doing it reliably.
The job id matters more than any other field. Titles get edited, locations change, descriptions get rewritten, and the same role is often reposted in several cities. If you de-duplicate on title plus company you will count one role three times and miss a quiet repost. If you de-duplicate on the id the platform assigns, you get a stable key that survives edits and tells you exactly when a posting appeared and disappeared. Everything below builds on that one decision.
What you can see without logging in
LinkedIn shows public job listings to visitors who are not signed in. The public search page returns a list of cards with a title, company name, location, posting date and a link to the listing. Each listing URL contains a numeric id. That id is what you store. The public listing page shows the description, seniority level, employment type and often the number of applicants as a rounded range. Salary is shown only when the employer entered it or when LinkedIn estimates it, and the estimate is not always present for logged-out visitors.
What a logged-out visitor cannot see is important too. You do not see who posted the role, you do not see the recruiter's name, and you do not see "Easy Apply" state consistently. You cannot filter on every facet that a signed-in user can. Some listings are only visible to members, and some employers pay for promoted placement, which changes the order of results but not the underlying set. The public view is also capped. Searching for one company and paging through results stops after a fixed number of pages, so a very large employer with thousands of open roles cannot be captured completely from the public search. For most competitor tracking that is fine, because you watch a handful of companies with tens or hundreds of roles, not a global employer with thousands.
LinkedIn's own rules on automated access are strict. The LinkedIn User Agreement prohibits scraping with a logged-in account and prohibits bypassing access controls. Any setup that logs in with a member account to collect data is a violation of that agreement and can get the account restricted. The setup described here only touches pages that are served to anyone on the open web, which is a different legal and technical situation, but you should read the agreement yourself and decide what is acceptable for your organisation.
Free sources that are often better
Before you collect anything from a job board, check whether the competitor publishes its own feed. Many companies run their career site on an applicant tracking system, and several of those systems expose public, documented endpoints that return every open role as JSON. Greenhouse publishes the Job Board API, which returns all live jobs for a board slug with a stable id per posting, no key required. Lever has a similar public postings endpoint. Workday, SmartRecruiters and Ashby career sites also return structured data that is easy to read.
When a competitor uses one of these systems, the ATS feed is the better source. It is complete, it includes internal ids and department fields, it updates the moment a role is opened or closed, and it carries no terms of service risk because the company publishes it on purpose. The job board copy of the same role may appear a day later, may be missing some roles that were never syndicated, and may include stale postings that the employer forgot to remove. Use the job board when the competitor has no accessible career feed, when you want to track many companies through one interface, or when you specifically want to know what the competitor chooses to advertise rather than what it has open internally. Those are different questions.
A second free source is Google's job search index. Employers who want their roles to appear in Google's job panel mark up each listing with the JobPosting structured data schema, which includes a datePosted, validThrough and an identifier. If you can fetch the competitor's career pages directly, parsing that markup gives you clean dates and ids without any third party at all.
A third option is manual. One person opens the competitor's career page every Monday and pastes the titles into a spreadsheet. For two or three competitors with a dozen roles each, this takes ten minutes and needs no tooling. The weakness is that nobody records the id, so reposts and edits are invisible, and the task quietly stops happening when the person is on holiday. If you only need a rough sense of direction, do this first and automate later.
The weekly run
The schedule itself is the easy part. Decide on a fixed day and time, and keep it. Weekly is the right cadence for most purposes. Daily runs produce a lot of noise, because listings flicker on and off when an employer edits them, and the signal you care about is the trend over months. Monthly runs miss short-lived postings entirely. Weekly catches almost everything that was open for more than a few days and gives you a clean series for charts.
Run the collection at a quiet time, early morning in the competitor's timezone, so you see the state after the previous working day's changes. Record the run timestamp with every row. If a run fails, re-run it the same day rather than skipping the week, because a gap in the series looks like a mass closure in the diff.
Most scraping platforms have a built-in scheduler. On Apify, for example, a schedule is a cron expression attached to an actor and its input, and each run writes to its own dataset. The Apify schedules documentation describes the format. Any cron job on a server, a GitHub Actions workflow on a cron trigger, or a cloud function with a timer does the same thing. The scheduler does not matter. What matters is that the input is identical every week: same company names, same location filter, same search terms. Change the input and the diff is meaningless until a new baseline is established.
De-duplication on the job id
Store each listing in a table keyed on the platform's job id. Keep at least these columns: job id, company, title, location, posting date as shown by the platform, the URL, the first run date you saw it, and the last run date you saw it. Every week, for each id in the fresh result, either insert it with first seen equal to today, or update last seen to today. Nothing is ever deleted.
The diff then falls out of two queries. New this week is every row where first seen equals the current run date. Closed this week is every row where last seen equals the previous run date and the id did not appear in the current run. Still open is everything with last seen equal to today and first seen before today. Age of a posting is today minus first seen. Average age of open roles per competitor is a useful number on its own, because a role that stays open for four months says something about either the role or the employer.
A few details make this robust. Treat the id as a string, not a number, because leading zeros and very long ids break in spreadsheets. Normalise the company name once, at insert time, so "Acme Inc." and "Acme" do not become two companies. Keep the title as it appeared on each run, and store title history if you want to detect a role being re-levelled from senior to mid, which happens more often than you would expect. If the same role is posted in five cities, you will see five ids. That is correct. Count them as five postings and group by title when you want to count roles.
Reposts are the one case where the id is not enough. Some employers close a posting and open a new one with a new id for the same role to reset the posting date and get back to the top of search. The id view shows one closure and one opening in the same week. If you want to catch this, compare the title and location of each new id with the closures from the same week, and flag exact matches as a probable repost. Do not merge them automatically. Keep the two ids and add a link column.
Where the data is incomplete
Be honest with whoever reads the output. The posting date shown on a job board is the date the listing was last published, not necessarily the date the role was opened. An edited listing may show a fresh date. Your own first seen column is more reliable for trend work than the platform's date, as long as your weekly series has no gaps.
Applicant counts, when present, are rounded and self-reported by the platform and should be read as a range. Salary is present only for a fraction of listings. Remote and hybrid labels are inconsistent across employers. Some roles are posted on the competitor's own site and never syndicated to any job board, so a job board view undercounts. Some roles are evergreen postings that collect applicants for a pipeline and never correspond to a real opening, so a job board view also overcounts. Neither error is large for most companies, but neither is zero, and the direction of the error differs by employer.
The public search also returns a limited number of results per query. Split large companies by location or by keyword so each query stays under the cap, and union the results before de-duplicating. The id key makes the union safe, because overlapping queries simply return the same id twice.
Reading the result
After four to six weeks the table starts to tell a story. A competitor that opens ten engineering roles in a month and closes none is building a team. A competitor whose average posting age jumps from thirty days to ninety days is struggling to fill, or has frozen hiring while leaving listings up. A sudden wave of closures without matching openings, especially across departments, is worth a closer look at that company's news. None of this is proof of anything. It is a public signal that costs almost nothing to collect, and its value comes entirely from the consistency of the weekly series and the discipline of keying on the id.
If you would rather not maintain the collector yourself, the LinkedIn jobs scraper on this site returns public listings with the job id, company, title, location, posting date and URL as structured rows, runs on a schedule and charges only for the results it delivers, so a weekly run for a few competitors stays small and predictable.
Want the signal instead of the raw filings? Get a free report preview. Prefer the tool to the write-up? Browse all data feeds or connect the free MCP server.