Use the internal Tumblr SVC API to access hidden (dashboard-only) blogs.

[MrEldritch]

After some research and experimentation, I've managed to find an alternative API that can be used to access hidden Tumblrs and posts that require a login to view. I've written a proof-of-concept for how to access the API and retrieve a JSON of very similar format to what the regular API uses here.

The resulting JSON is almost, but not quite, identical to the usual format; the primary differences are simply that some fields are given different names, and that the "blog" field that gives general information about the Tumblr is given as part of each "post" rather than as a separate top-level field. (Also, it doesn't include the total number of posts for that blog). There are a few other differences, but for the most part all the information is still there and in the same place; so while a separate parser would need to be written, it would not be very separate. (It's also quite straightforward to detect whether a tumblr is hidden or not, and therefore which parser you'd need to use - you just try to access it at the normal blogname.tumblr.com URL, and see if it redirects you to a URL containing "login_required" or "dashboard/blog/".)

I've been working on trying to fully integrate this functionality myself, but it's been slow going as I'm still working out how your own script works exactly. (It doesn't help that I'm primarily a Python 3 user, myself); I could use some help.

[MrEldritch]

As an example, here's what the JSON looks like when you request the first 50 posts using the SVC API via the proof-of-concept method I included, and here's what you get from the ordinary Tumblr v2 API you're already using.

I have not yet worked out a good solution for archiving liked posts from hidden or explicit tumblrs.

[MrEldritch]

Actually, nevermind, archiving liked posts from hidden tumblrs is piss-easy; you can just use the ordinary v2 API url. Works perfectly fine. I just assumed it wouldn't because, for instance, trying to access a hidden tumblr with api.tumblr.com/v2/blog/{blog-identifier}/info?api_key={key} doesn't work.

But even for dashboard-only tumblrs, api.tumblr.com/v2/blog/{blog-identifier}/likes?api_key={key} works just fine. (Although note that you have to make sure your mainblog is set to 'explicit' , or it'll only return SFW likes)

[johanneszab]

You could at least have said that you just re-wrote code from TumblThree into python instead of claiming you managed to find an alternative to the api. You even have the same "user name" suggestions returned from the Tumblr SVC, which I highly doubt will ever be suggested a second time, next to the same parameter set, method names, redirection detection and user agent and all the other suggestions here.

No hard feelings, it's all open source, and I'd like to see to spread, but I actually sat there a few days figuring out what request to send and how to answer them.

[MrEldritch]

@johanneszab You're right on that count. To be clear, this was just a follow-on from some earlier discussion on another issue where I'd been explicitly stating that I was trying to reverse-engineer how TumblThree did it; I simply felt I'd put together enough progress that I wanted to make a new issue to boost it in visibility.

I will say that this also took me several days, because I had literally never read a line of C# or worked with Python 2 or done anything involving HTTP requests before, so trying to figure out how TumblThree and tumblr-utils actually worked well enough to figure out what the relevant bits were, and what they were doing, (and going down many, many dead ends and overreaches before I realized that I was doing this really inefficiently and should just put something minimal together to make it easier for the real programmers) ended up filling up quite a few sleepless nights.

...although if I'd actually spent those nights sleeping, it probably would have occurred to me that I could have just asked you how it worked, which I have only just now realized.

So yeah, it certainly felt like a heroic personal effort after spending a couple dozen hours learning enough through trial-and-error to get to this point[1], but I fully acknowledge that it only took that long and required so much effort because I'm an extreme novice - this would have taken, like, two hours for anyone mildly competent. You did the part that would have been actually difficult for someone who actually knew what they were doing.

[1] just to be clear, I don't mean it took me that long to write that particular code - there was a lot of stuff in-between, like exploring the details of precisely how the json formats worked for the different kinds of requests and what the differences were, or trying to figure out if "Hidden" meant "dashboard-only" or "explicit" or both, or... anyway, I could have written this particular proof-of-concept a couple days ago, it just took me a long time of hammering at this problem before I realized that I was not the best person for this job, I was doing it really inefficiently, and I should just try and put together something really minimal to show the person who already knew how their program worked.

[MrEldritch]

@johanneszab ...also, that proof-of-concept code originally had some comments literally reading #This bit is pure voodoo to me; I just copied these requests from TumblThree's code.

I thought those were still in there, but it looks like I accidentally deleted them while I was cleaning up and re-organizing that part of the code into its own procedure.

Anyway, yes, you certainly deserve the lion's share of the credit for this.