Cedric from over-blog asked me a question today: how often does Mobitype refresh feeds published on other servers?
A rather simple question, but the answer is quite complex.
In fact, Mobitype refresh policy changed many times since first releases. The current feed base is about 20,000 feeds, we cannot refresh them too often, having server resources limited. But it is discouraging to consistently have stale feeds in the database ;)
So, first of all, all feeds are divided into two groups. There are alive feeds and dead feeds. A feed is alive when it has at least one subscriber, or it has at least 100 consultations per week.
For all alive feeds, an average time spacing between posts is calculated within last several posts (posts in future are not counted, since there are users who timestamp their posts in late 2099). Having this average spacing as A, the refresh time for the feed is calculated as A/20. Afterwards, it is bounded to be between 30 minutes and 1 day, to be reasonable.
The refresh time for dead feeds is about once every several days.
So when you browse lists of feeds, like tops, favorites or text search, the local database copy of the feed is displayed. However, when user clicks on "View post", the local copy is verified. If it has not been refreshed for more then 2 hours, the live request is being sent to the remote server at the time of the page rendering, and the feed eventually updated.