Error at the end of fitting

[cgreenberg]

File "run_corex.py", line 26, in <module>
   out.fit(mat2)
File "/home/charles/LinearCorex/linearcorex/linearcorex.py", line 134, in fit
   last_tc = self.tc  # Save this TC to compare to possible updates
 File "/home/charles/LinearCorex/linearcorex/linearcorex.py", line 176, in tc
   return self.moments["TC"]
TypeError: 'bool' object has no attribute '__getitem__'

Happened after I tried a long run - it seemed to finish fitting but then die at the end. Hard to find exactly the cause since it runs fine with smaller data sets (this one had size 126654x10000 with n_hidden=500 and gpu=True)

[gregversteeg]

Thanks for pointing this out. I have run into this before, and I know where to look for the problem. I’ve been on vacation and now conference, but will try to implement a fix soon.

…

-Greg

On Aug 17, 2017, at 10:57 AM, Charles Greenberg ***@***.***> wrote: File "run_corex.py", line 26, in <module> out.fit(mat2) File "/home/charles/LinearCorex/linearcorex/linearcorex.py", line 134, in fit last_tc = self.tc # Save this TC to compare to possible updates File "/home/charles/LinearCorex/linearcorex/linearcorex.py", line 176, in tc return self.moments["TC"] TypeError: 'bool' object has no attribute '__getitem__' Happened after I tried a long run. Hard to find exactly the cause since it runs fine with smaller data sets (this one had size 126654x10000 with n_hidden=500 and gpu=True — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH8ph507-78E6CJKApnJW_-uBpXj16ofks5sY4_jgaJpZM4O5p-L>.

[gregversteeg]

This took me much longer than I thought! I found that there was an error in a formula that re-scaled solutions after each annealing step. This sometimes caused invalid solutions leading to a numerical error. The bug was subtle because most of the time the re-scaling has little effect.
Anyway, let me know if you get a chance to try again with the new code. (Others have had the same issue, so I hope to get some confirmation that this fixes the bug soon.)

[pantaray]

I tried with the most recent patch to linearcorex but still get an error, albeit of a different flavor. Here's the output:

Linear CorEx with 44 latent factors
3409 iterations to tol: 0.000010
1236 iterations to tol: 0.000010
LinearCorex/linearcorex/linearcorex.py:259: RuntimeWarning: divide by zero encountered in divide
  m["Y_j^2"] = self.yscale ** 2 / (1. - m["uj"])
LinearCorex/linearcorex/linearcorex.py:271: RuntimeWarning: divide by zero encountered in log
  + 0.5 * np.sum(np.log(1 - m["uj"]))
LinearCorex/linearcorex/linearcorex.py:271: RuntimeWarning: invalid value encountered in log
  + 0.5 * np.sum(np.log(1 - m["uj"]))
LinearCorex/linearcorex/linearcorex.py:275: RuntimeWarning: invalid value encountered in sqrt
  m["X_i Y_j"] = m["rho"].T * np.sqrt(m["Y_j^2"])
LinearCorex/linearcorex/linearcorex.py:278: RuntimeWarning: invalid value encountered in log
  m['I(Y_j ; X)'] = 0.5 * np.log(m["Y_j^2"]) - 0.5 * np.log(self.yscale ** 2)
LinearCorex/linearcorex/linearcorex.py:292: RuntimeWarning: divide by zero encountered in divide
  grad = self.ws / rj
Warning: step size becoming too small
Error... updates giving invalid solutions?
Linear CorEx with 8 latent factors
263 iterations to tol: 0.000010
119 iterations to tol: 0.000010
46 iterations to tol: 0.000010
13 iterations to tol: 0.000010
2 iterations to tol: 0.000010
36 iterations to tol: 0.000010
6 iterations to tol: 0.000010
Linear CorEx with 1 latent factors
105 iterations to tol: 0.000010
82 iterations to tol: 0.000010
34 iterations to tol: 0.000010
16 iterations to tol: 0.000010
2 iterations to tol: 0.000010
3 iterations to tol: 0.000010
0 iterations to tol: 0.000010
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
run_corex.py in <module>()
    344     # Exract 1st layer and associated relevant information
    345     cx = corexes[0]
--> 346     mi = cx.mis
    347     ws = np.abs(cx.ws)
    348     n_factors, N = mi.shape

linearcorex.py in mis(self)
    186     @property
    187     def mis(self):
--> 188         return - 0.5 * np.log1p(-self.moments["rho"]**2)
    189
    190     def clusters(self):

TypeError: 'bool' object has no attribute '__getitem__'

[gregversteeg]

Thanks Stefan, I managed to construct a test case that reliably produces the error. I can see exactly what the issue is, it should take me another day or two to solve. -Greg

…

On Oct 12, 2017, at 5:37 AM, Stefan Fuertinger ***@***.***> wrote: I tried with the most recent patch to linearcorex but still get an error, albeit of a different flavor. Here's the output: Linear CorEx with 44 latent factors 3409 iterations to tol: 0.000010 1236 iterations to tol: 0.000010 LinearCorex/linearcorex/linearcorex.py:259: RuntimeWarning: divide by zero encountered in divide m["Y_j^2"] = self.yscale ** 2 / (1. - m["uj"]) LinearCorex/linearcorex/linearcorex.py:271: RuntimeWarning: divide by zero encountered in log + 0.5 * np.sum(np.log(1 - m["uj"])) LinearCorex/linearcorex/linearcorex.py:271: RuntimeWarning: invalid value encountered in log + 0.5 * np.sum(np.log(1 - m["uj"])) LinearCorex/linearcorex/linearcorex.py:275: RuntimeWarning: invalid value encountered in sqrt m["X_i Y_j"] = m["rho"].T * np.sqrt(m["Y_j^2"]) LinearCorex/linearcorex/linearcorex.py:278: RuntimeWarning: invalid value encountered in log m['I(Y_j ; X)'] = 0.5 * np.log(m["Y_j^2"]) - 0.5 * np.log(self.yscale ** 2) LinearCorex/linearcorex/linearcorex.py:292: RuntimeWarning: divide by zero encountered in divide grad = self.ws / rj Warning: step size becoming too small Error... updates giving invalid solutions? Linear CorEx with 8 latent factors 263 iterations to tol: 0.000010 119 iterations to tol: 0.000010 46 iterations to tol: 0.000010 13 iterations to tol: 0.000010 2 iterations to tol: 0.000010 36 iterations to tol: 0.000010 6 iterations to tol: 0.000010 Linear CorEx with 1 latent factors 105 iterations to tol: 0.000010 82 iterations to tol: 0.000010 34 iterations to tol: 0.000010 16 iterations to tol: 0.000010 2 iterations to tol: 0.000010 3 iterations to tol: 0.000010 0 iterations to tol: 0.000010 --------------------------------------------------------------------------- TypeError Traceback (most recent call last) run_corex.py in <module>() 344 # Exract 1st layer and associated relevant information 345 cx = corexes[0] --> 346 mi = cx.mis 347 ws = np.abs(cx.ws) 348 n_factors, N = mi.shape linearcorex.py in mis(self) 186 @Property 187 def mis(self): --> 188 return - 0.5 * np.log1p(-self.moments["rho"]**2) 189 190 def clusters(self): TypeError: 'bool' object has no attribute '__getitem__' — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH8phx9vWY5DuHE6HgaAx9qqBFu9qLRCks5srggbgaJpZM4O5p-L>.

[gregversteeg]

I just uploaded some new numerical fixes. Let me know if that works for you, and I'll close the issue.

[pantaray]

Thank you very much, Greg! Works like a charm now :)