Skip to content

Chapter 4 - Getting Twitter data into Kafka - TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType' #3

Open
@06GitHub

Description

@06GitHub

Chapter 4 - Getting Twitter data into Kafka - TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

When executing following cli command (chapter 4 page 116)

python3 ~/stream.py -j | ./kafka-console-producer.sh --broker-list localhost:9092 --topic tweets

Following error is generated

File "~/stream.py", line 18, in init
self.limit = int(numtweets)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

It seems that -n parameter is mandatory for stream.py, see line 18 below

18--> self.limit = int(numtweets)

No error is generated when -n parameter is set

python3 ~/stream.py -j -n 1000 | ./kafka-console-producer.sh --broker-list localhost:9092 --topic tweets

Any workaround/solution possible ? Thanks.

stream.py

-- coding: utf-8 --

import tweepy
import os
import json
import argparse

consumer_key = os.environ['TWITTER_CONSUMER_KEY']
consumer_secret = os.environ['TWITTER_CONSUMER_SECRET']
access_key = os.environ['TWITTER_ACCESS_KEY']
access_secret = os.environ['TWITTER_ACCESS_SECRET']

class EchoStreamListener(tweepy.StreamListener):
def init(self, api, dump_json=False, numtweets=0):
self.api = api
self.dump_json = dump_json
self.count = 0
18--> self.limit = int(numtweets)
super(tweepy.StreamListener, self).init()

def on_data(self, tweet):
    tweet_data = json.loads(tweet)
    if 'text' in tweet_data:
        if self.dump_json:
            #print tweet.rstrip()
            print (tweet.rstrip())
        else:
            #print tweet_data['text'].encode("utf-8").rstrip()
            print (tweet_data['text'].encode("utf-8").rstrip())

        self.count = self.count+1
        return False if self.count == self.limit else True

def on_error(self, status_code):
    return True

def on_timeout(self):
    return True

[snip]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions