greta_sitrep() fails, despite github install_great_deps() reporting no errors

[rogerssam]

Hi Nick, my colleague Max attended your workshop on Wednesday, and was having problems installing greta dependencies on his MBP (M2 chip). I have been trying to assist with the installation issues this morning.

It seems that somehow greta_sitrep() is not detecting the installed packages properly? I'm not exceptionally familiar with python environment setups myself, but here is some output from what we've tried and what was returned.

> install_greta_deps()
✔ Python modules installed!                                              
• To see full installation notes run:
• `greta_notes_conda_install_output()`
• To see any error messages, run:
• `greta_notes_conda_install_error()`
✔ Installation of greta dependencies is complete!
• Restart R, then load greta with: `library(greta)`

All appears to have worked correctly. When checking error messages (greta_notes_conda_install_error()) it only reports updates available for pip, no installation errors.

However, greta_sitrep() doesn't think tensorflow is available:

> greta_sitrep()
✔ python (v3.8) available     
✔ TensorFlow Probability (v0.21.0) available  
✖ TensorFlow not available        
✔ greta conda environment available            
✖ Initialising python and checking dependencies, this may take a moment.…
Warning message:
We have detected that you do not have the expected python packages
setup.
You can set these up by running this R code in the console:
`install_greta_deps()`
Then, restart R and run:
`library(greta)`
(Note: Your R session should not have initialised Tensorflow yet.)
For more information, see `?install_greta_deps` 

However, checking via reticulate shows it does appear to be installed and available:

> reticulate::conda_list()
           name           python
1          base           /Users/xxxxx/Library/r-miniconda-arm64/bin/python
2 greta-env-tf2           /Users/xxxxx/Library/r-miniconda-arm64/envs/greta-env-tf2/bin/python
3  r-reticulate           /Users/xxxxx/Library/r-miniconda-arm64/envs/r-reticulate/bin/python


> reticulate::py_list_packages("greta-env-tf2")
                        package    version
1                       absl-py      2.1.0
2                    astunparse      1.6.3
3                    cachetools      5.3.2
4                       certifi 2023.11.17
5            charset-normalizer      3.3.2
6                   cloudpickle      3.0.0
7                     decorator      5.1.1
8                       dm-tree      0.1.8
9                   flatbuffers    23.5.26
10                         gast      0.5.4
11                  google-auth     2.27.0
12         google-auth-oauthlib      1.2.0
13                 google-pasta      0.2.0
14                       grpcio     1.60.0
15                         h5py     3.10.0
16                         idna        3.6
17                        keras     2.15.0
18                     libclang     16.0.6
19                     Markdown      3.5.2
20                   MarkupSafe      2.1.4
21                    ml-dtypes      0.2.0
22                        numpy     1.26.3
23                     oauthlib      3.2.2
24                   opt-einsum      3.3.0
25                    packaging       23.2
26                     protobuf     4.23.4
27                       pyasn1      0.5.1
28               pyasn1-modules      0.3.0
29                     requests     2.31.0
30            requests-oauthlib      1.3.1
31                          rsa        4.9
32                          six     1.16.0
33                  tensorboard     2.15.1
34      tensorboard-data-server      0.7.2
35                   tensorflow     2.15.0
36         tensorflow-estimator     2.15.0
37 tensorflow-io-gcs-filesystem     0.34.0
38             tensorflow-macos     2.15.0
39             tensorflow-metal      1.1.0
40       tensorflow-probability     0.23.0
41                    termcolor      2.4.0
42            typing_extensions      4.9.0
43                      urllib3      2.2.0
44                     Werkzeug      3.0.1
45                        wrapt     1.14.1
                            requirement
1                        absl-py==2.1.0
2                     astunparse==1.6.3
3                     cachetools==5.3.2
4                   certifi==2023.11.17
5             charset-normalizer==3.3.2
6                    cloudpickle==3.0.0
7                      decorator==5.1.1
8                        dm-tree==0.1.8
9                  flatbuffers==23.5.26
10                          gast==0.5.4
11                  google-auth==2.27.0
12          google-auth-oauthlib==1.2.0
13                  google-pasta==0.2.0
14                       grpcio==1.60.0
15                         h5py==3.10.0
16                            idna==3.6
17                        keras==2.15.0
18                     libclang==16.0.6
19                      Markdown==3.5.2
20                    MarkupSafe==2.1.4
21                     ml-dtypes==0.2.0
22                        numpy==1.26.3
23                      oauthlib==3.2.2
24                    opt-einsum==3.3.0
25                      packaging==23.2
26                     protobuf==4.23.4
27                        pyasn1==0.5.1
28                pyasn1-modules==0.3.0
29                     requests==2.31.0
30             requests-oauthlib==1.3.1
31                             rsa==4.9
32                          six==1.16.0
33                  tensorboard==2.15.1
34       tensorboard-data-server==0.7.2
35                   tensorflow==2.15.0
36         tensorflow-estimator==2.15.0
37 tensorflow-io-gcs-filesystem==0.34.0
38             tensorflow-macos==2.15.0
39              tensorflow-metal==1.1.0
40       tensorflow-probability==0.23.0
41                     termcolor==2.4.0
42             typing_extensions==4.9.0
43                       urllib3==2.2.0
44                      Werkzeug==3.0.1
45                        wrapt==1.14.1

As you can see, TF and TF probability are both listed there as available in that environment, but apparently not visible to greta for some reason.

Any suggestions?

[njtierney]

Hi @rogerssam - Thanks for this! Really appreciate the extra information on the R session / python packages etc.

One clue here that is interesting is that tensorflow-probability is version 0.21.0 in the greta sitrep, but listed below it is version 0.23.0 - do you have two versions installed, perhaps?

[rogerssam]

Hi @rogerssam - Thanks for this! Really appreciate the extra information on the R session / python packages etc.

One clue here that is interesting is that tensorflow-probability is version 0.21.0 in the greta sitrep, but listed below it is version 0.23.0 - do you have two versions installed, perhaps?

Not sure about TF-P, but when listing the installations of TF there were 64 of them, of various different versions. I imagine there were several TF-P versions there as well, but I didn't specifically look.

[rogerssam]

I've been having a look at the code for greta_sitrep() to understand what it's doing. Can you tell me where tf is assigned as used for getting the version in this line:

greta/R/utils.R

Line 66 in f59c1f0

tf_version <- suppressMessages(tf$`__version__`)

[rogerssam]

Hi @njtierney, any further thoughts on this?

[njtierney]

@rogerssam sorry I missed this!

Can you tell me where tf is assigned as used for getting the version in this line:

tf is assigned at

https://github.com/greta-dev/greta/blob/tf2-poke-tf-fun/R/zzz.R#L3

How is installation progressing from here? Recent changes to tf2-poke-tf-fun branch might mean that this will install now. Let me know how you go!

[njtierney]

Hi @rogerssam - I've revamped the greta installation process, if you can try installing from

devtools::install_github("greta-dev/greta")

And run

library(greta)
# set a log file to save the installation information into
write_greta_install_log('greta-logfile.html')
reinstall_greta_deps()

Let me know how you go? I'm going to close this issue for the moment but let me know how things go.

[rogerssam]

Thanks @njtierney, this seems to be sorted now.

[njtierney]

So glad to hear it! :)