Display http headers

[CaptainQuirk]

Hi,

Thanks for fx ! It's a great tool to explore json without leaving the terminal.

I guess this is a bit out of scope (and possibly complicated to achieve) but I think it would be great to be able to pipe the result of a curl request with the -i flag to fx and have the headers displayed at the top of the viewport.

Thanks for your time !

[antonmedv]

curl -i https://medv.io | fx -rs | fx

[CaptainQuirk]

Hi,

Thank you for your quick response !

There may have been a misunderstanding because the command you proposed returns a json array of headers, which is not what I had in mind

I was wondering if the current fx viewport could contain :

• HTTP Headers at the top, possibly with syntax highlighting and the yanking feature
• Json data right underneath, explorable like it is now

Something like the following :

HTTP/1.1 200 OK
Content-Length: 11910

{
  "result": {      }
}

[antonmedv]

I see. I nice idea but I'm not sure how to be able to detect those kinds of headers. It should be integrated into our json parser which will be tricky.

[CaptainQuirk]

I'm not aware of the component you use for the presentation layer, nor of the global architecture of fx, but maybe it means that the viewport has to be composed of two different parts, one dedicated to json with its attached parser (jq ?) and an optional one for headers ?

[antonmedv]

Problem is not with displaying the headers. Problem is with parsing them. How to distinguish them from JSON? Or invalid JSON?

[CaptainQuirk]

Wouldn't a regular expression help to determine that the given data is the combination of http headers and a body separated by a null line, like it's stipulated in rfc1945 section-4.1 ?
In any case, if the regular expression is not the way to go, wouldn't an additionnal flag do the trick ?
What about an --http flag that instructs fx that the given data is indeed http ?

[antonmedv]

I guess we can add a custom parser which checks if a first line is from curl, and starts parsing headers.

HTTP/1.1 200 OK

Would you like to try to implement such a feature? Note parser should be implemented in both Go parser and JS parser:

fx/json.go

Line 20 in 1952074

func parse(data []byte) (head *node, err error) {

fx/npm/index.js

Line 272 in 1952074

function* parseJson(gen) {

[CaptainQuirk]

Hi !

I have no notion of Go whatsoever unfortunately

[andrei-korshikov]

pipe the result of a curl request with the -i flag to fx and have the headers displayed

No, -i is not enough. Let's parse --verbose:D — I want to see not only response headers, but request ones too (and some protocol-related messages). And I prefer to have all that stuff at the bottom, by the way:

#! /bin/sh

set -o errexit
set -o nounset

readonly curl_stdout='curl_stdout'
readonly response_body='response.json'
readonly transfer_info='transfer_info.json'

curl_version=$(curl --version | grep --perl-regexp --regexp='(?<=^curl )\S+' --max-count=1 --only-matching)

curl --disable \
    'https://wiki.archlinux.org/api.php' \
    --data 'action=query' \
    --data 'meta=siteinfo' \
    --data 'format=json' \
    --data 'formatversion=latest' \
    --get \
    --output "${response_body}" \
    --silent \
    --stderr - \
    --user-agent "bot (User:Andrei_Korshikov) curl/${curl_version}" \
    --verbose \                                           
    >"${curl_stdout}"                                     
                                                          
printf '{"transfer info": {\n' >"${transfer_info}"        
sed --regexp-extended --expression='s_\r__ ;# remove CR (^M) from headers' "${curl_stdout}" \
    | grep --perl-regexp --regexp='^(> .|< .|\* .HTTP)' \
    | sed --regexp-extended --expression='
        s_"_\\"_g               ;# escape double quotes
        s_^(. \S+):_\1_         ;# remove colons after header names
        s_^(. \S+) _"\1": "_    ;# shape first word as a key, start a value
        s_.*_&",_               ;# finalize a value
        ' \
        >>"${transfer_info}"
printf '}}\n' >>"${transfer_info}"

cat "${response_body}" "${transfer_info}" | FX_THEME=2 fx

The sample is almost real—in my real script I use --data "@$1", all other lines are the same.

And, to be clear—of course, I do not ask to implement --verbose parsing in fx:)

[andrei-korshikov]

Frankly, I'm not sure if parsing a plain-text output (-i) is a good (reliable) idea. We can kindly ask curl to provide all response headers in JSON:

curl 'https://wiki.archlinux.org/api.php?action=query&meta=siteinfo&format=json&formatversion=latest' \
  --silent --write-out '%{header_json}' | fx