More places to look, and your city first

The first job of a job search tool is to show you the jobs. That sounds too obvious to write down, and this week it turned out to be the hard part twice over.

The first problem is coverage. Most of the jobs worth applying for are posted first on the employer’s own careers page, and an aggregator gets them days later or never. So the site now reads ten more of the systems companies use to run those pages.

The second problem was quieter and worse. Search for a software job in Chicago and, until Thursday night, seven of the hundred results were in Chicago. The other ninety-three were remote jobs, newer and louder, that had pushed every local listing off the page before you could see it.

You asked for Chicago. You got the internet. The fix was not more sources but a better order.

Where the jobs come from now

There are two ways to get more jobs onto the page. You can buy them, or you can read the places companies already publish them. This week was almost all the second.

Ten more company-board systems. The site already read Greenhouse, Lever and Ashby. It now also reads SmartRecruiters, Workable, Rippling, Gem, Pinpoint and JobScore, plus four that are mostly used in Europe: Recruitee, Personio, Teamtailor and Manatal. Every one is read through the vendor’s own public job-board interface, the one built so companies can show their jobs on other sites. Each vendor’s limits are kept: Workable is read one request every second and a half, and JobScore at most once an hour per board, because its terms say so.

Nothing is scraped. That line was drawn months ago and it held: no reading of web pages that were not published for this, no logging in as anyone, no getting round a bot check. One narrow exception, approved this week, helps the site find which companies post on Workable. It runs once a month, at a slow pace, and never reads a job posting.

Two new search sources, on our own keys. 4 Day Week and Rise are searched live, and each is credited with a link back. Rise asked for that link to send them traffic they can see, and a fix the same evening made sure it does. A third, Startup Jobs, is built and switched off until its terms on storing listings are confirmed.

Built, and switched off on purpose. The directory that reads thousands of company boards every day runs on the test site, where it holds 1,386 boards and about 66,600 jobs. On the live site it is off until each board has been reviewed by a person, because a board answering under a company’s name proves nothing about who posted to it.

What the gates caught

Before a release goes out, two reviewers read everything that changed and try to break it, and a third works through the site the way a person would. This week they earned their place.

A spreadsheet that could stop the site. The importer for job spreadsheets had been taught a new rule this week: a quote mark in the middle of a value is just a character, the way spreadsheets read it. The code that checked for that re-read the whole value at every quote. A file with two million quote marks in one cell, small enough to pass every size limit, would have held the server for about five minutes, and anyone with an account could send one. It now reads the file once, in 78 milliseconds.

Reviewing the fixes, not just the findings. Each fix went back to a reviewer. Twice the review of a fix found something new: approval rules that trusted numbers the employer’s own board supplied, and a board that could come back live with nobody looking after its name was taken over. Both were fixed before anything shipped.

Your city, first. (Shipped Oct 1, 23:05 UTC, as Release 4.2.) The Chicago problem came from how the page picked its hundred results: newest first, from every source at once. Remote job boards post all day, so they always won. Now the site keeps everything the sources return and orders a city search by place: your area first, then remote work, then jobs that name no place, then everywhere else. On the test site, a fresh search for Denver now shows fifty Colorado jobs, all at the top, where the old order showed a handful. The reviewers caught that the first version read “Toronto, ON, CA” as Los Angeles and “Bengaluru, KA, IN” as Indianapolis. It reads them as Canada and India now.