« Micro-Optimism: The Slow Drip of Disappointment |
Main
| NowPublic.com's Community News Building »
Pushing Google Reader's Limits
Don't try this at home.
I bumped up my Google Reader subscriptions last night from around 350 to 7570 for a work project. Fortunately, the blogs are very low volume so the increase in blog consumption wouldn't be that significant.
However, it turns out that Google Reader had some problems with this:
Or, more specifically, Firefox or Safari would both time out before opening my subscriptions.
Luckily, Google offers a subscription management page under the Settings tab so I was able to filter for, and unsubscribe from, the newly added large batch.
Now I'm looking for other ideas on how to make this work. I tried using Yahoo Pipes to generate a single RSS feed from a 7000+ feed OPML file but ran into time-out issues with that as well.
Do you have any ideas worth trying? I'm sure Google Reader can handle the post volume - just not the individual feed volume - so how can I generate an aggregate feed of a large OPML file?
TrackBack
TrackBack URL for this entry:
http://www.technologyevangelist.com/cgi-bin/mt-tb.fcgi/1446
2. Posted by: Vladimir on June 3, 2008 2:16 PM:
Hi,
Is it posible to create 14+ Yahoo Pipes for 500 feed each( or other amount what yahoo can handle) and then join created feeds?
3. Posted by: bluenorway.org on June 3, 2008 7:00 PM:
wget, cat, dom parse to extract sortables, sort sortable, que targets, extract/consolidate?
(assumes you want to sort.)
magpie_fetch, simplepie (bad), etc etc
4. Posted by: Jon on June 3, 2008 9:05 PM:
i guess the low tech solution would be to change the timeout period in firefox?
5. Posted by: Jon on June 3, 2008 9:27 PM:
I've spent a small amount of time looking for a way to increase that period and cannot locate it. So maybe not.
6. Posted by: strm on June 4, 2008 3:26 PM:
How about building various variants of the opml file with something between 1-2000 feeds in each one? Just delete parts of it and save it as a new file.
7. Posted by: Kyle LeNeau on June 4, 2008 4:12 PM:
Have you checked out http://feedblendr.com/. It might be able to do what your looking for, I'm not sure what their limits are though (if any). Or have you looked at any external (desktop application based) readers that can consume the OPML file or the thousands of feeds.
8. Posted by: steve jenings on June 6, 2008 5:39 AM:
Aren't the time out periods adjustable?
9. Posted by: Stephan Miller on June 20, 2008 9:38 PM:
I ran into the same issues with Yahoo pipes and an OPML of 600 feeds. Feedhub.com is the solution I finally found. I alllow you to build multiple river type feeds from an OPML file. It also has a meme tracking feature. I found this solution after searching for almost three hours. I wasn't sure what I was looking for, just the results I wanted. One feed from multiple feeds.
| 
1. Posted by: justin on June 3, 2008 11:37 AM:
Sounds like a job for a custom script (python, php, perl...) running as a cron job somewhere. Just have it aggregate the feeds, then spit out one gigantic RSS or ATOM file, subscribe to that. Keep in mind that it might take a good chunk of memory to process 7500 RSS feeds.