tag:blogger.com,1999:blog-9099779.post7595703690321534346..comments2024-03-23T23:09:17.426+01:00Comments on StalkR's Blog: Twitter ArchiverStalkRhttp://www.blogger.com/profile/15113480981262771031noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-9099779.post-44553815293547147302012-11-24T10:52:58.067+01:002012-11-24T10:52:58.067+01:00Hi mate,
It is slow because it has to make 1 requ...Hi mate,<br /><br />It is slow because it has to make 1 request/user minimum (to check if there are new tweets) and request limit is low.. unfortunately.<br /><br />The timeline is rebuilt from scratch but don't worry this is just a concatenation and sort by time of all tweet files created by twitter-archiver, and these are not created every time: they resume automatically and only new tweets are saved.<br /><br />The reason it is rebuilt from scratch is because you might have new followers/following, so new tweets with an older date need to be inserted in the timeline. Similarly, you may have removed followers/following and their tweets needs to be removed from timeline.<br /><br />If you ignore these cases, then yes you can build an append-only timeline that you can "tail -f". The algorithm can be as simple as append from last known tweet id.<br /><br />Let me know if you need help doing that,<br />Cheers.StalkRhttps://www.blogger.com/profile/15113480981262771031noreply@blogger.comtag:blogger.com,1999:blog-9099779.post-43840498540455936652012-11-15T21:04:35.981+01:002012-11-15T21:04:35.981+01:00Hi mate, thank you for this nice post!
I am looki...Hi mate, thank you for this nice post!<br /><br />I am looking for the best way to save/archive all my tweets (mine and people I follow) to a log file continuously (using an interval) avoiding duplicates.<br />When I use :<br />"twitter friends -d -t -r -R 30 --format default >> twitter.log"<br />Duplicate entries are included since the same tweets are logged more than once.<br /><br />I tried your db.sh script (https://github.com/StalkR/misc/blob/master/twitter/db.sh) which is mentioned above too, but when the followers/following users are many it takes hours to create the timeline since the hourly limit for requests is reached and actually the file is created each time from scratch. <br />So "tail -f" doesn't help to check and maybe then grep the latest tweets.<br /><br />Any ideas/suggestions? Thanks in advance!Anonymousnoreply@blogger.com