Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chapter 4 - Getting Twitter data into Kafka - TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType' #3

Open
06GitHub opened this issue Aug 17, 2017 · 0 comments

Comments

@06GitHub
Copy link

Chapter 4 - Getting Twitter data into Kafka - TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

When executing following cli command (chapter 4 page 116)

python3 ~/stream.py -j | ./kafka-console-producer.sh --broker-list localhost:9092 --topic tweets

Following error is generated

File "~/stream.py", line 18, in init
self.limit = int(numtweets)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

It seems that -n parameter is mandatory for stream.py, see line 18 below

18--> self.limit = int(numtweets)

No error is generated when -n parameter is set

python3 ~/stream.py -j -n 1000 | ./kafka-console-producer.sh --broker-list localhost:9092 --topic tweets

Any workaround/solution possible ? Thanks.

stream.py

-- coding: utf-8 --

import tweepy
import os
import json
import argparse

consumer_key = os.environ['TWITTER_CONSUMER_KEY']
consumer_secret = os.environ['TWITTER_CONSUMER_SECRET']
access_key = os.environ['TWITTER_ACCESS_KEY']
access_secret = os.environ['TWITTER_ACCESS_SECRET']

class EchoStreamListener(tweepy.StreamListener):
def init(self, api, dump_json=False, numtweets=0):
self.api = api
self.dump_json = dump_json
self.count = 0
18--> self.limit = int(numtweets)
super(tweepy.StreamListener, self).init()

def on_data(self, tweet):
    tweet_data = json.loads(tweet)
    if 'text' in tweet_data:
        if self.dump_json:
            #print tweet.rstrip()
            print (tweet.rstrip())
        else:
            #print tweet_data['text'].encode("utf-8").rstrip()
            print (tweet_data['text'].encode("utf-8").rstrip())

        self.count = self.count+1
        return False if self.count == self.limit else True

def on_error(self, status_code):
    return True

def on_timeout(self):
    return True

[snip]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant