June 21, 8:40 AM PST
Pinboard is down for most users right now. We're running off of backup but many people report not being able to reach the site at all.
Our main database server is still unreachable and so is our ISP (!). I've moved the service to our backup DB server, but since it's in the same data center I can't guarantee that it won't also be affected by whatever is plaguing our hosting provider.
Your bookmark data is safe, but none of us can get to it right now. I'm provisioning a new server in a different locat ion, but it will take a while to load the backup into it.
Next status update at 12:30 PM PST. Please see our Twitter stream (@Pinboard) for blow-by-blow updates.
-Maciej Ceglowski
Pinboard Status
June 21, 12:38 PM PST
Just received word from our hosting company that they were raided by the FBI who pulled some racks of equipment. No word on whether our server was among those machines, or whether it is just offline. In the meantime the site is running on a backup server with reduced capabilities (see below). All bookmarks are intact.
June 21, 12:26 PM PST
Here's more detail on what's going on:
We have three big servers with one hosting company (DigitalOne). Two of them are online, but our main database server is unreachable. Basically it is cut off from the world because of network problems at our provider.
A bunch of us who host at DigitalOne are trying to track them down right now for a status update, but have not had any luck. Last I heard from them (Monday night PST) they were fixing a router problem.
In the meantime, I'm running the site off of our backup database server. This server is weak, so to keep it from bogging down I have turned off the API, RSS feeds, the popular and recent page, as well as all user tag pages. I will turn these services back on as soon as possible.
Search and global tag pages are also unavailable, since they run on the unreachable machine.
One thing that does work is our export page, and if you're feeling nervous about your data I encourage you to use it. We have fresh backups (from one day ago) safely stored on S3, but in these matters you can't be too careful.
I will update this page with news, but for the latest please follow our Twitter feed.
June 21, 15:41 PM PST
I've turned user+tag pages back on, as well as bookmark imports. Still no further word from our hosting providers. I'm provisioning a new server, but setup will take at least a day to complete. No RSS feeds or API access until then unless it turns out our server wasn't one of the ones taken in the raid.
June 22, 8:42 AM PST
Service has stabilized and I've been able to turn on the API. Still not working: some archive links, search, global tag pages, RSS, tag clouds, user stats. I hope to get these back online during the day.
DigitalOne has confirmed that our server was one of the ones taken during the FBI raid. I have no reason to believe it had anything to do with us, but unfortunately these blade servers pack many to a single box.