Making requests without running afoul of Twitter's rate limits

For this recipe, we are going to modify the function created in the previous recipe, Pulling Twitter user profiles, in order to avoid hitting the dreaded Twitter API rate limits.

Getting ready

You will again need the list of followers' and friends' IDs from the previous recipe, Pulling Twitter user profiles, as well as the authenticated Twython object.

How to do it...

The following function demonstrates how you can retrieve a set of Twitter users' profiles in a rate-limit-aware fashion:

In [25]: import time 
    ...: import math 
 
In [26]: rate_limit_window = 15 * 60 #900 seconds 
 
In [27]: def pull_users_profiles_limit_aware(ids): 
    ...: users = [] 
    ...: start_time = time.time() 
    ...: # Must look up users in 
    ...: for i in range(0, len(ids), 10): 
    ...: batch = ids[i:i + 10] 
    ...: users += twitter.lookup_user(user_id=batch) 
    ...: calls_left = float(twitter.get_lastfunction_header('x-rate-limit-remaining...