Documentation for the Feed2tweet project

Feed2tweet parses a RSS feed, extracts the last entries and sends them to Twitter. You’ll find below anything you need to install, configure or run Feed2tweet.

Guide

How to install Feed2tweet

From PyPI

$ pip3 install feed2tweet

From sources

  • You need at least Python 3.4.

  • On some Linux Distribution setuptools package does not come with default python install, you need to install it.

  • Install PIP:

    $ wget https://bootstrap.pypa.io/get-pip.py -O - | sudo python3
    
  • Install setuptools module:

    $ wget https://bootstrap.pypa.io/ez_setup.py -O - | sudo python3
    

Alternatively, Setuptools may be installed to a user-local path:

$ wget https://bootstrap.pypa.io/ez_setup.py -O - | python3 - --user
  • Untar the tarball and go to the source directory with the following commands:

    $ tar zxvf feed2tweet-1.0.tar.gz
    $ cd feed2tweet
    
  • Next, to install Feed2tweet on your computer, type the following command with the root user:

    $ python3 setup.py install
    $ # or
    $ python3 setup.py install --install-scripts=/usr/bin
    

Configure Feed2tweet

As a prerequisite to use Feed2tweet, you need a Twitter app. Log in Twitter, go to https://apps.twitter.com, create an app and generate the access token.

In order to configure Feed2tweet, you need to create a feed2tweet.ini file (or any name you prefer, finishing with the extension .ini) with the following parameters:

[twitter]
consumer_key=ml9jaiBnf3pmU9uIrKNIxAr3v
consumer_secret=8Cmljklzerkhfer4hlj3ljl2hfvc123rezrfsdctpokaelzerp
access_token=213416590-jgJnrJG5gz132nzerl5zerwi0ahmnwkfJFN9nr3j
access_token_secret=3janlPMqDKlunJ4Hnr90k2bnfk3jfnwkFjeriFZERj32Z

[cache]
cachefile=/home/user/feed2tweet/cache.db
cache_limit=10000

[rss]
uri: https://www.journalduhacker.net/rss
uri_list: /home/user/feed2tweet/rsslist.txt
tweet: {title} {link}
title_pattern: Open Source
title_pattern_case_sensitive: true
no_uri_pattern_no_global_pattern=true

[hashtaglist]
several_words_hashtags_list: /home/user/feed2tweet/hashtags.txt

For the [twitter] section:

  • consumer_key: the Twitter consumer key (see your apps.twitter.com webpage)
  • consumer_secret: the Twitter consumer secret key (see your apps.twitter.com webpage)
  • access_token: the Twitter access token key (see your apps.twitter.com webpage)
  • access_token_secret: the Twitter access token secret key (see your apps.twitter.com webpage)

For the [cache] section:

  • cachefile: the path to the cache file storing ids of already tweeted links. Absolute path is mandatory. This file should always use the .db extension.
  • cache_limit: length of the cache queue. defaults to 100.

For the [rss] section:

  • uri: the url of the rss feed to parse
  • uri_list: a path to a file with several adresses of rss feeds, one by line. Absolute path is mandatory.
  • tweet: format of the tweet you want to post. It should use existing entries of the RSS fields like {title} or {link}. Launch it with this field empty to display all available entries.
  • {one field of the rss feed}_pattern: takes a string representing a pattern to match for a specified field of each rss entry of the rss feed, like title_pattern or summary_pattern.
  • {one field of the rss feed}_pattern_case_sensitive: either the pattern matching for the specified field should be case sensitive or not. Default to true if not specified.
  • no_uri_pattern_no_global_pattern: don’t apply global pattern (see above) when no pattern-by-uri is defined in the uri_list. Allows to get all entries of a rss in the uri_list because no pattern is defined so we match them all. Defaults to false, meaning the global patterns will be tried on every rss in the uri_list NOT HAVING specific patterns and so ONLY entries from the specific uri in the uri_list matching the global patterns will be considered.

For the [hashtaglist] section:

  • several_words_hashtags_list: a path to the file containing hashtags in two or more words. Absolute path is mandatory. By default Feed2tweet adds a # before every words of a hashtag.

List of rss feeds

Simple list of rss feeds

With the parameter uri_list, you can define a list of uri to use. Starting from 0.10, Feed2tweet is now able to match specific patterns for each of the rss feeds from this list. Consider the following rss section of the configuration file:

[rss]
uri_list=/home/john/feed2tweet/rsslist.txt
tweet={title} {link}

Now let’s have a look at the =/home/john/feed2tweet/rsslist.txt file:

https://www.journalduhacker.net/rss
https://carlchenet.com/feed

Each line of this file is a url to a rss feed. Pretty simple.

Match specific patterns of rss feeds in the uri_list files

Starting with Feed2tweet 0.10, you can now use specific pattern matching for uri in the uri_list file to filter some of the rss entries of a rss feed. Lets modify the previous file:

https://www.journalduhacker.net/rss|title|hacker,psql https://carlchenet.com/feed|title|gitlab

Each line of this file starts with an uri, followed by a pipe (|), followed by the name of the available section to parse (see below), again followed by a pipe (|), followed by patterns, each pattern being separated from the other one by a semi-colon (,).

In the example file above wee get every rss entries from the feed available at https://www.journalduhacker.net/rss where a substring in the title section of this entry matches either “hacker” or “psql”. Specific patterns are not case sensitive. For the second line, we match every rss entries from the feed available at https://carlchenet.com/feed where a substring in the title section of this entry matches “gitlab”.

Consider every entries of a rss feed from a uri in the uri_list file

Starting from Feed2tweet 0.10, it is now possible to get all entries from a rss feed available in the uri_list file. You need an option to deactivate the global pattern matching for uri in the uri_list NOT having specific patterns:

[rss]
...
no_uri_pattern_no_global_pattern=true

In you rsslist.txt, just don’t give anything else than the needed feed url to get all the entries:

https://www.journalduhacker.net/rss|title|hacker,psql https://carlchenet.com/feed|title|gitlab https://blog.linuxjobs.fr/feed.php?rss

The last line of the file above only has the url of a rss feed. All entries from this feed will be tweeted.

How to display available sections of the rss feed

Starting from 0.8, Feed2tweet offers the –rss-sections command line option to display the available section of the rss feed and exits:

$ feed2tweet --rss-sections -c feed2tweet.ini
The following sections are available in this RSS feed: ['title', 'comments', 'authors', 'link', 'author', 'summary', 'links', 'tags', id', 'author_detail', 'published'].

Use Feed2tweet

After the configuration of Feed2tweet, just launch the following command:

$ feed2tweet -c /path/to/feed2tweet.ini

Run Feed2tweet on a regular basis

Feed2tweet should be launche on a regular basis in order to efficiently send your new RSS entries to Twitter. It is quite easy to achieve with adding a line to your user crontab, as described below:

@hourly feed2tweet -c /path/to/feed2tweet.ini

will execute feed2tweet every hour. Or without the syntactic sugar in the global crontab file /etc/crontab:

0 * * * * johndoe feed2tweet -c /path/to/feed2tweet.ini

Test option

In order to know what’s going to be sent to Twitter without actually doing it, use the –dry-run option:

$ feed2tweet --dry-run -c /path/to/feed2tweet.ini

Debug option

In order to increase the verbosity of what’s Feed2tweet is doing, use the –debug option followed by the level of verbosity see [the the available different levels](https://docs.python.org/3/library/logging.html):

$ feed2tweet --debug -c /path/to/feed2tweet.ini

Populate the cache file without posting tweets

Starting from 0.8, Feed2tweet offers the –populate-cache command line option to populate the cache file without posting to Twitter:

$ feed2tweet --populate-cache -c feed2tweet.ini
populating RSS entry https://www.journalduhacker.net/s/65krkk
populating RSS entry https://www.journalduhacker.net/s/co2es0
populating RSS entry https://www.journalduhacker.net/s/la2ihl
populating RSS entry https://www.journalduhacker.net/s/stfwtx
populating RSS entry https://www.journalduhacker.net/s/qq1wte
populating RSS entry https://www.journalduhacker.net/s/y8mzrp
populating RSS entry https://www.journalduhacker.net/s/ozjqv0
populating RSS entry https://www.journalduhacker.net/s/6ev8jz
populating RSS entry https://www.journalduhacker.net/s/gezvnv
populating RSS entry https://www.journalduhacker.net/s/lqswmz

How to display available sections of the rss feed

Starting from 0.8, Feed2tweet offers the –rss-sections command line option to display the available section of the rss feed and exits:

$ feed2tweet --rss-sections -c feed2tweet.ini
The following sections are available in this RSS feed: ['title', 'comments', 'authors', 'link', 'author', 'summary', 'links', 'tags', id', 'author_detail', 'published'].

Plugins

Starting from 0.9, Feed2tweet now supports plugins. Plugins offer optional features, not supported by default. Optional means you need a dedicated configuration and sometimes a dedicated external dependencies. What you need for each module is specified below.

InfluxDB

The InfluxDB plugin allows to store already published tweets in a InfluxDB database.

Install the InfluxDB plugin

To install Feed2tweet with the InfluxDB plugin, execute the following command.

From scratch:

# pip3 install feed2tweet[influxdb]

Upgrading from a previous version, execute the followin command:

# pip3 install feed2tweet[influxdb] --upgrade

Configuration

Below is the block of configuration to add in your feed2tweet.ini:

[influxdb]
;host=127.0.0.1
;port=8086
user=influxuser
pass=V3ryS3cr3t
database=influxdb
measurement=tweets
  • host: the host where the influxdb instance is. Defaults to 127.0.0.1
  • port: the port where the influxdb instance is listening to. Defaults to 8086
  • user: the user authorized to connect to the database. Mandatory (no default)
  • pass: the password needed to connect to the database. Mandatory (no default)
  • database: the name of the influxdb database to connect to. Mandatory (no default)
  • measurement: the measurement to store the value into. Mandatory (no default)

License

This software comes under the terms of the GPLv3+. It was previously under the MIT license. See the LICENSE file for the complete history of the license and the full text of the past and current licenses.

Authors

Carl Chenet <chaica@ohmytux.com>

Indices and tables