Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR: Update the file properties on load #24

Open
elvarb opened this issue Jun 14, 2024 · 20 comments
Open

FR: Update the file properties on load #24

elvarb opened this issue Jun 14, 2024 · 20 comments
Assignees
Labels
enhancement New feature or request

Comments

@elvarb
Copy link

elvarb commented Jun 14, 2024

Is your feature request related to a problem? Please describe.
I want to utilize Shodan to document what it knows about certain assets there and would like to utilize dataview and other properties tools to create the bigger picture.

Describe the solution you'd like
When an API Request block is triggered to update it should be able to update the file frontmatter with the values it returns the same way as you would format the output.

Describe alternatives you've considered
Other alternatives I have looked at is using Templater and javascript, but this plugin is a much better fit.

Additional context
Example code from the documentation

url: https://api.chess.com/pub/player/hikaru/stats
show: chess_daily -> last -> rating, chess_daily -> best -> rating
format: <p>Last game: {}</p> <strong>Best game: {}</strong>

Could then be used like this as an example

url: https://api.chess.com/pub/player/hikaru/stats
show: chess_daily -> last -> rating, chess_daily -> best -> rating
format: <p>Last game: {}</p> <strong>Best game: {}</strong>
properties: last-game: {}, last-rating: {} 
@Rooyca Rooyca self-assigned this Jun 14, 2024
@Rooyca Rooyca added the enhancement New feature or request label Jun 14, 2024
@Rooyca
Copy link
Owner

Rooyca commented Jun 14, 2024

We could even remove the {} in the properties. Don't you think:

...
properties: last-game, last-rating

We may have a problem with the properties types. I'll take a look at this.

@elvarb
Copy link
Author

elvarb commented Jun 18, 2024

I'm not sure if that is necessary to think about property types as this is all text. The only case where that might be an issue is when the api request fails or it returns some value that is unexpected.

For example you expect to get a true/false to use as a boolean property but instead get "undefined" or "null".

Similar function as is in the plugin for "notify-if" could be used to ensure that the values are as expected.

@elvarb
Copy link
Author

elvarb commented Jun 18, 2024

Also I see that the feature has been added and I did some tests and it works really well 👍

Only one issue which I think is more with Obsidian than how the plugin works.

When the api returns a list of ports as

"ports": [ 
   80, 
   443 
]

It correctly creates the file properties as

ports:
- 80
- 443

But sadly Obsidian does not like a list of numbers being listed that plainly, it works great for text. To make it appear correctly it must appear as

ports:
- "80"
- "443"

When manually working with the properties in Obsidian it will automatically put " around numbers when used in lists. So maybe it is by design.

@elvarb
Copy link
Author

elvarb commented Jun 18, 2024

A great enhancement to the properties feature would be allowing for writing the properties as internal links.

For example this simple block queries the Shodan free InternetDB API for the file name which is an ip address and writes it as a property.

url: https://internetdb.shodan.io/{{this.file.name}}
show: ip
properties: ip

An enhancement could be implemented that would turn the ip property into an internal link (not sure if this would work from a coding perspective but some other indicator of it being an internal link would work as well)

url: https://internetdb.shodan.io/{{this.file.name}}
show: ip
properties: [[ip]]

The properties would then be as an internal link

ip: [[127.0.0.1]]

@Rooyca
Copy link
Owner

Rooyca commented Jun 18, 2024

A great enhancement to the properties feature would be allowing for writing the properties as internal links.

I like this idea. I'll check what I can do 😄

@elvarb
Copy link
Author

elvarb commented Jun 19, 2024

A great enhancement to the properties feature would be allowing for writing the properties as internal links.

I like this idea. I'll check what I can do 😄

Fantastic work. It works perfectly for single returns but not with list items, so for example when the show function returns a list, for example

["something","else"]

@Rooyca
Copy link
Owner

Rooyca commented Jun 19, 2024

It works perfectly for single returns but not with list items

I tested it with numerical lists and it worked. Didn't test it with strings (my bad). I'll have a look at it.

@Rooyca
Copy link
Owner

Rooyca commented Jun 19, 2024

Hey @elvarb, I found something and I'd like your opinion.

For now properties only can be used when show is present (as a way of preventing entire JSON in the front-matter) but I create an simple API that return ['Hello', 'world'] and couldn't use properties because show wasn't present.

My question is: should that limitation be removed? should properties be used whenever the user want to?

@elvarb
Copy link
Author

elvarb commented Jun 20, 2024

The way the plugin works is that show pulls the values from the JSON response and properties put those into the designated property with the values used in show.

  • Right way to display the response would be to follow it with format, meaning show becomes redundant.
  • Similarly using properties outputs the formatted response into the file properties.
  • If everything works then having show output anything at all is not required.

Perhaps it would be good to have a dedicated debug function that would output everything, or perhaps some function the allows the user to select how the output is shown.

  • debug - show every output, raw json, output of show, output of all functions
  • normal - default behaviour - show only output of format and all errors, if no format is defined only show errors
  • suppress - show nothing, even errors

The use case I see with suppress is having the api request block at the top of the file that would just output into the file properties making it transparent. And another huge benefit of putting the values in the properties is that you can pull them into the markdown file by using for example the following:

- **Company**: `= this.company`
- **Email**: `= this.email`

@Rooyca
Copy link
Owner

Rooyca commented Jun 20, 2024

Now I am even more lost haha.
I plan to remove the limitation on properties. If a user wants to store the full JSON in the front-matter, that's up to them.

Regarding the debug option, I'm not sure I see the use for it.

@elvarb
Copy link
Author

elvarb commented Jun 21, 2024

But how would it work if you have a json file with a value you want to use in a property and the names are not the same. Lets say you have in the json something like "name" but you want to store that under the property "company" ?

@Rooyca
Copy link
Owner

Rooyca commented Jun 21, 2024

Lets say you have in the json something like "name" but you want to store that under the property "company" ?

That would work. They don't need to have the same name.

@elvarb
Copy link
Author

elvarb commented Jun 27, 2024

That means you have to use show with properties, so requiring show is fine.

What could be changed to clarify the usage of show, that it would never be rendered unless something like debug is used. The normal usage to render the data from the response would be to identify the data with show and then rendering it out with format and storing it with properties. Would that make sense?

@Rooyca
Copy link
Owner

Rooyca commented Jun 27, 2024

I think it's not necessary to use format, because properties do not store styles.

@elvarb
Copy link
Author

elvarb commented Jun 28, 2024

I mean that you would use format when you want the output to be rendered when viewing the file. Show would never output anything and only be for format and properties to know what values to use.

@Rooyca
Copy link
Owner

Rooyca commented Jun 28, 2024

I'm not going to lie to you. I'm a little lost with what you're saying. Maybe I'm wrong but I think the way you describe things are the way they are working right now.

@elvarb
Copy link
Author

elvarb commented Jul 1, 2024

Sorry about that, now with examples.

The public InternetDB API shodan has for the ip 8.8.8.8 is called with

https://internetdb.shodan.io/8.8.8.8

This returns the following json

{
  "cpes": [],
  "hostnames": [
    "dns.google"
  ],
  "ip": "8.8.8.8",
  "ports": [53],
  "tags": [],
  "vulns": []
}

Using it with Request API

url: https://internetdb.shodan.io/8.8.8.8
show: ip
format: # {}
render
  • This will set up the format for the ip address and then render it
  • Works as expected 👍

Now we add properties as well

url: https://internetdb.shodan.io/8.8.8.8
show: ip
properties: address
format: # {}
render
  • This will set up the format for the ip address and then render it
  • This will write the ip value in the json in the address property.
  • Works as expected 👍

Now I want to only save the properties

url: https://internetdb.shodan.io/8.8.8.8
show: ip, ports
properties: address, openports
  • This will write the ip value in the json in the address property.
  • This will also output the ip value in a block

Would be great to have the option of having no output at all as saving the value in the file properties and rendering them out from there inline gives much more flexibility.

| Google DNS Server| Values  |
| --- | --- |
| IP Address | `= this.address` |
| Open ports | `= this.openports` |
Google DNS Server Values
IP Address = this.address
Open ports = this.openports

@Rooyca
Copy link
Owner

Rooyca commented Jul 1, 2024

Now I understand 😄. You are absolutely right, that's why you suggested adding the debug flag, right?

A small "problem" (to call it somehow) is that by not showing anything the literal code-block disappears and it is easy to forget that there is a code-block in the document (I say this from my own experience. I had a document full of code-blocks that didn't show anything and the only way I noticed it was when I activated source mode).

Maybe I could add a counter (in the status bar) of code-blocks in the current document. Or what other idea do you have?

@elvarb
Copy link
Author

elvarb commented Jul 2, 2024

Yes thats correct. Also would help when trying to walk through the json data in the show option, debug could then show both the raw json and the data you pull from it with show.

That is a good point about everything being hidden, it is never good to have something like this completely invisible to the user.

A counter in the status bar would be a good solution, especially if it can provide some more feedback to the user. Perhaps one small icon per code block and then have the icon color coded to indicate the results.

  • Red for errors connecting to the API
  • Orange or Yellow for errors in parsing or anything regarding working with the response
  • Green or Gray for successful execution of the codeblock
  • Blue or Gray if the codeblock was not executed because the setting told it not to run (Issue#29)

Would be awesome to have this as part of the plugin settings, both to enble/disable it and also to customize the colors as I think some would like to have it match the theme they are using.

@Rooyca
Copy link
Owner

Rooyca commented Jul 3, 2024

It's a good idea to implement colors for the answers. Although I don't know how easy it would be to add something like that.

We've talked about so many things that I don't even know where to start haha. I'll see what I can do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants