devilry-log

Setting up the environment

* Downloaded pycharm for linux
* Installed python-pip
** sudo apt-get python-pip
* Installed python-virtualenv
** sudo apt-get python-virtualenv
* Created a virtualenv environment in ~/django
** ~/virtualenv django
* Created a python files folder
** mkdir django/src
* Cloned devilry-django from private fork
** git clone https://github.com/ilyakh/devilry-django.git
* Created a public key for github
** https://github.com/settings/ssh
* Created an alias to activate and switch to the devilry environment
** emacs ~/.bash_aliases
** + alias devilry="source ~/devilry/bin/activate && cd ~/devilry/src"

Configuring devilry demo
** sudo apt-get install build-essential python-dev python-virtualenv
** sudo pip install pycrypto
** sudo pip install fabric
** mkdir -p ~/.buildout/eggs ~/.buildout/dlcache
** cd devenv
** fab setup_demo

Configuring tools
* Created an alias for devilry
** alias devilry="cd ~/devilry-django/devenv/bin/ && django_dev.py"
* Installed ipython
** sudo pip install ipython
* Installed django-extensions for easier model access
** sudo pip install django-extensions
* Added django-extensions to the development settings
** emas ~/devilry-django/devenv/settings/development_settings.py
** (+) INSTALLED_APPS += [ ... 'django-extensions', ]
* Installed OpenJDK Runtime
* Set environment variable PYCHARM_JDK to Oracle JDK path
* Downloaded and unpacked pycharm into ~/Desktop/
* Install watchdog
** sudo pip install watchdog

Creating the nodeadmin app
* Create /src/devilry_nodeadmin/devilry_nodeadmin
* Create template and static folders
** mkdir template static 
* Create python files
** touch __init__.py models.py urls.py
* Create a setup.py file, fill out package details just like in other apps
* Create README.rst
* Run bin/buildout from ./devenv/
** cd ./devenv/ && ./bin/buildout
* Create rest API directory
** cd ./src/devilry_nodeadmin/devilry_nodeadmin/
** mkdir rest
** cd rest
** touch urls.py __init__.py
* Fill out devilry_nodeadmin/urls.py and devilry_nodeadmin/rest/urls.py
*  Create a 'clean' branch
* Added deafult app template in /templates/devilry_nodeadmin/app.django.html
* Added a static folder in /static/devilry_nodeadmin/

Clean branch now contains the clean version of devilry for further development of app:devilry_nodeadmin.

* Nodes are generic abstracts that stand for organizational hierarchy.
* Node administrators need an overview of the subjects connected to the nodes they administer.
* Node.where_is_admin function yields a queryset of all nodes that param:user is admin in.
  >>> u = DevilryUserProfile.objects.get( user__username='ifiadmin' )  
  >>> n = Node.where_is_admin( u )
  [<Node: duckburgh.ifi>] 
* Each node may or may not be a parentnode of a semester
  >>> Subject.objects.all()[0].parentnode
  <Node: duckburgh.ifi>
* To choose all subjects that are under the authority of a nodeadmin, I run a filter query: Subject.objects.filter( parentnode=n )
  >>> Subject.objects.filter( parentnode=n )
  [<Subject: duck1010>, ... , <Subject: duck6000>]
* (!) Note that the actual administrator is not a User object, but a DevilryUserProfile instance.
* Created an app-specific app.js file in static, based on devilry_frontpage
* To start developing without the app-specific css, I disabled the link tag in /templates/app.django.html via django comment-tag. The django tag system, while being unable to find the css resources and/or folder will bitch like crazy.
* Added the i18n url to the urls.py.
* Removed the missing widgets and packages that were mentioned in console's error log from app.js (cleaned up js).

ExtJS Stores
* I started reasearching ExtJS stores to create node, subject and period overview.
* Created a static/devilry_nodeadmin/app/ hierarchy
** mkdir app
** cd app
** mkdir store model view controller
* Began working on model/store elements
* Model is an abstraction directly coupled with rest-api. It fetches objects posted out 
** touch model/Node.js
* Trying to decide what kind of features (fields, properties) a Extjs Node model might need. Node as an instance, has following properties:
** for i in dir(Node):
**     if not i.startswith("_"):
**     	  print i
* Which of these need to be accessible by Extjs? etag, short_name, long_name, id.
* Since the rest API will only return the nodes that the user is admin of, then the admin and ownership related fields are not of any essence
* Researching ManyToMany-relationships in Extjs.
** http://www.sencha.com/blog/ext-js-4-anatomy-of-a-model/
* Every django model has the 'object'-property which contains a manager with query-like refernces to every serialized object in the object-source (the database). Ext js has its own object source which is django's rest api, and extjs' stores serve exactly as the object managers in django. In django terms, to create a custom manger for Extjs, create a Store: for instance User.js model may have a BannedUsers.js store. This is used for both the objects themselves and their respective relationships.
* To be able to pass the unserialized data to Extjs, django must reuse the existing rest-framework utilities: ModelResource and for ListModelView. 
* Fields in ModelResource are defined in a static property fields (a tuple of string) and may be overridden by functions of the same name.
* To pass multiple objects to ListModelView, override get_queryset (return a subset of Node.objects.all). 
* The child of ListModelView must have 'resource' property and may need 'permissions' set to ( IsAuthenticated, ).
* Every View child is added to the urls.py with as_view() call.
** [+] url( r'nodes/', AttachedNodes.as_view() )
* [?] How to handle many-to-many relationships in rest-framework?

2012-12-12
* Researching the interaction of controllers and views.
* Integrating the stores and models into the controllers. 
* Displaying views for a model of choice (Node).
* widget.[...] namespace contains all views defined with alias, like 'widget.nodeadmin.overview' will be accessible as xtype 'nodeadmin.overview'.
* Trying to find out why does imports fail, when I try to require a view from app.js in NodeAdmin.
* Got the first output in a custom element of the app.js
* Now attempting to move this element into a widget of its own.
* Ext js seems to be failing to import the external files. Collectstatic does not solve the problem.
* (!) I have created a broken directory tree in /static/. It lacked one level of separation. The right hierarchy should be: /static/app/[view, controller, store, ...]. In other words, the devilry_nodeadmin folder should contain: app.js, app-all.js and '/app/view/', '/app/controller/' and all other ext-specific folders.
* Succeded in making the app.js fetch a widget from a view in its own file.

2012-12-13
* Attempting to create a store for Node, with a proxy inside of it.
* I probably should rename the AttachedNodes to RelatedNodes.
* Trying to get the json data from rest-framework in a view via. store.
* Ext.js stores need to be loaded (store.load(r, options, success)) with a callback as a parameter.
* Effective practice: test things in the main file first and only then separate the code into controller, view and model files.
* The ajax reponse data can be accessed by
** var a = Ext.create( "devilry_nodeadmin.store.AttachedNodes" ); a.load({ callback: function(records,options,success) { console.log( records[0].raw ) } })

2012-12-14
* I have isolated the store rest data fetching error to the store itself. Attempting to correct the code.
* Assign a temporarty property data: [ ... ] to a Store to check out if it is configured correctly and whether the error lies in the rest queries.
* Attempted to interpolate a template over a json data.
* Created a simple Grid-like structure that displays the list of nodes fetched from from json rest source.
* Attempting to create custom templates.
* { xtype: ... } is used to instantiate an element; an element like container can have its own subtree of elements inside the property "items" (a list of objects).
* Created custom templates. Can be nested and contain headers and footers.

2012-12-21
* 'where_is_admin' requires UserProfile, while 'where_is_admin_or_superadmin' requires User
* Strings in the ModelResource's property fields are names of functions that define the respective data that will be returned in those fields.
* To fetch an instance with the earlist/latest start date or time you have to first find it via aggregation and only then find in the subset of instances.

2012-12-30
* Attempting to add hierarchy to the rest-responses.
* It is possible to inherit from one serializers in order to display relations to the same model. Not sure if this is a correct way of doing it, but I'm attempting anyway.
* Fetching both parents and children is data replication, in the final version I will use only one of this properties, and fetch the other (if necessary in its own rest request).
* Trying to create a separated view to fetch children from.
* The code of ParentRelatedNode and ChildRelatedNode is repetitive now. See of both are necessary or can be generalized.
* I couldn't choose all the nodes that do not have a parent as the base for the node list, because some administrators have access to nodes lower in the hierarchy.

2012-12-31
* mr. developer causes a lot of trouble with dirty packages just because of static files (I assume). It's errors are not presented clearly in the run of fabric setup_demo, but usually they fail because some git pull url error. As the packages are already in your developlment branch and do not need to be updated, remove their mr.developer sources from development-base.cfg. This is the only thing that will make the thing run.
* To represent the hiererchy: the list returned by where_admin_or_superadmin contains nodes, some of them are children of other nodes in the list. In a special rest-resource I will exclude the nodes with parents in the same list, forcing the list to contain only the top-level nodes. Their children can be later fetched via. rest queries either at application start, or through navigation.
* Remind the maintainer to fix the where_is_admin and where_is_admin_or_superadmin inconsistency: the prior takes in the DevilryUserProfile as the parameter, while the latter uses django's User.
* I am fetching the results of where_is_admin_or_superadmin and exlude all the rows where the parent is already in the response. This way I am filtering out direct children and can build the node tree more efficiently.
* Remember to check is_admin on all subsequent nods in the hierarchy.
* Returning one query with unlimited depth doesn't work in Ext.js, since I can't seem to find any recursive relationship definitions except hasMany which only goes one level down. Will look at the templating syntax and see if that can be fixed, if not, I will go for the least efficient solution and fetch the information about each child node for every tree level.
* Attempting to flatten the object hierarchy by creating a breadcrumb-like path for each subject-terminated branch.
* Another way of doing it is to display only the terminal nodes and augment their description with a path: it will require a rest view that returs the opposite of the currect exlude approach.


2012-03-01

* Routes override the standard browser navigation in request/response style, by replacing the http requests with a javascript event, this keeping the processing of the request in the client side. 
* My goal is to request a list of child nodes depending on the position in the navigation tree from the said client side js.
* To do this I have set up: 1. events in the constructor-method of the app-controller 2. event-handling functions in the body of the controller 3. _setupRoutes() function and its contents.
* Mind that router is a foreign party script that is not a part of ext js standard distridution.
* Created UrlLookup.js; the file seems to be used to inject the ids and pks into the urls to fetch particular objects by rest queries.
* When the application starts the Router should by default direct the defaultNodeList view.
* There's something wrong in my View definitions. I'm probably lacking some default methods in order to define a view completely. Will try to extend a Panel.
* After testing the initiation, it seems like the inheritance from view works and the default node overview is initiated.
* The view is rendered but seems to be empty. Testing whether the store initializations words when:
** store: 'devilry_nodeadmin.store.RelatedNodes'
* The store contains data, but isn't rendered. Trying to assign emptyText parameter to see if it will be used. 
* The empty text is not display. Trying renderTo. No effect.
* Trying to fetch store via StoreManager. Fails.
* Trying configs 'autoRender', 'autoShow'; no effect.
* 'cls' option in every View corresponds to the specific css class that will be applied to the component and its children.
* [!] Reason for views not displaying node contents: the view that specifically defines the 'store' parameter overrides something in the previous class, so the store itself needs to be instantiated and set to this.store _inside the initComponent method_. Otherwise, the tepmplate will be rendered, but its elements would be ommited.
** initComponent: function() {
**     this.store = Ext.create( "devilry_nodeadmin.store.RelatedNodes" )
**     this.callParent();
** }
* [x] [?] Should I use Router from extjsextras, or the one from Ext.js' 4.0? Transition.
* [x] [?] In order to enable Routes, I had to alter the urls.py of the django-application, to allow /nodes/:node_id subdirectory. This is not done on subjectadmin, but yet it works! Why?
* Solved: the routes fake the url-pattern by adding a # directory like this: the proper configuration of my node/:node_id path would be in full 'devilry_nodeadmin/#/node/2. Tested and works.
* Attempting to pass the 'node_id' parameter to the store and replace the contents of primaryContentContainer.
* Its probably best to rethink the rest approach and do not treat children and parent nodes differently, they should all be rendered and treated the same, yet the app must be tree level aware.
* Attempting to pass a node id to the store.
* Successfully passed the 'node_pk' parameter into the store. The parameter itself is assigned as a property on instantiation via xtype, and becomes accessible in the this-scope of the view, which passes it onto store through its initComponent-method in following manner:
** initComponent: function() {
**     this.store = Ext.create( ...
**     this.store.proxy.url = Ext.String.format( '/url/{0}', this.node_pk );
**     this.callParent(arguments);
** }
* Relevant areas of improvement: breadcrumbs with nodes' short_name; terminal nodes display subjects.
* Nested objects in templates must start with their own template scope tag. Following template will display the parent id followed by its child:
** <tpl for="."><tpl for="parent">{id}</tpl>child</tpl>
* Inconsistency in tpl tags often leads to Uncaught TypeError
* [?] [x] There seems to be a certain issue with the url dispatcher: if the argument in the regexp string is the same as a field of a model (or resource), and the target of the url is a descendant of ListModelView, then it will automatically filter out the objects by pk no matter what happens inside the overriden get_queryset-function.


2012-01-05

* As the navigation functions to some extend, I will solve the issue of terminal nodes (and the issue of top-level node operations) by creating a secondary content pane that will contain details on each level of the nagivation tree.
* Managed to split up the layout of the app into two columns: one for the navigation, and one for the details about each node. The correct configuration is: { container/border, { container/center/column { container/east|west} }
* When using the InstanceModelView for READ ONLY, remember to reduce the allowed methods to:
** allowed_methods = ['get'] 
* [?] [x] Inquire if it is ok to follow the convention and call a store and a view the same only deferring in the uppercase/lowecase first letter.
* Give the events the event-like names. Right now they describe the data they serve, not the event that selects the data.
* Can swap regions of the node browser and the node detail overview.
* Creating a link back to the parent node from a child view.
* Remember to use i18n and gettext
* Must redesign the boilerplated rest to allow going back to parent from a terminal node (terminal nodes return empty json-responses, so there is no refernce to which parent they come from: solutions to this is to switch from a list of children to a node with its own 'hasMany' children relation.
* The 'go to parent link', and parents in themselves have a potential caveat: they lead the user away from the nodes he has permission to administer. So on each up-step through the hierarchy one is ought to doublecheck, if the user actually owns the node.
* The whole parent node issue can be easily fixed by the breadcrumbs. 
* Attempting to create a list of Subjects and other multiply-related objects in the rest framework.
* Examine how the Subject-models are implemented in the other apps of the devilry.
* Remember to treat the pagination appropriately.
* [?] Splitting into Ext js panels vs. native html-composition
* Handle the url-editing attempts: add error message for missing route: "the node is either not found, out of scope or requiring higher priveleges"
* Consider appending an element (to recreate hierarchy) on navigation, instead of replacing it by setPrimaryContent.


2012-01-07
* If your 'fab bootstrap' fails to install sphinxbuilder due to 'Value Error: unknown locale', add following lines to your .bash_profile and run source ~/.bash_profile. 
** export LC_ALL=en_US.UTF-8
** export LANG=en_US.UTF-8
* Use bin/fab -l to see a list of defined tasks
* In order to integrate the application I had to add the devilry_nodeadmin in devilry_settings.default_urls, devilry_settings.default_settings, development_base.cfg for buildout: once in eggs, and once again in source prefixed by fs (filesystem).
** [sources]
** ...
** devilry_nodeadmin = fs devilry_nodeadmin
* There seems to be some confusion in Header.js in regards to the old 'administrator' role.
* Fixed by changing the 'navclass' property of the devilryheader element residing in this.viewport.
* [x] [!] Correct text in 'How administrator rights work': 'eighter' -> 'either'. Probably a rewrite of the whole sidebar on 'administrator rights' would do the trick.
* No error message if the date is edited and is set prior to the start date.
* Saved url-listing as API guide in devenv/urls.txt
* Suggest making the runserver by django_devserver to be optional.


2012-01-08
* I seem not to be getting access to bootstrap at all. Probably need to install Ruby and SASS.
* So point of controls is to initialize references and through them, render certain events on particular events. The one I am examining now at this point also initiates the store and renders in in a view.
* [x] [?] How is the ubuntu-font handled and what are the fallbacks in case it is not embedded and loaded dynamically.
* The question of how to inject an id into a store proxy's url arises again, but this time in the context of a controller. The solution seems to be a function inside each view corresponding to the url parameter. In subjectadmin this is done in following manner:
** devilry_subjectadmin:    
**  setAssignment: function(assignment_id) {
**    this.proxy.url = 
**      Ext.String.format(this.proxy.urlpatt, assignment_id);
**  },
* And again, what is 'urlpatt'? It doesn't sound like something from stdlib! Here's an explanation extracted from the bellows of 'RelatedStudentRo.js':
** urlpatt: D...s.DEVILRY_URLPATH_PREFIX 
**   + '/devilry_subjectadmin/rest/relatedstudent_assignment_ro/{0}/',
** url: null, // We use baseurl to dynamically set the url from urlpatt
* 'baseurl' seems to be defined in an abstract store that RelatedStudentRo inherits from. Not sure how to interfere with its inner workings, or whether I should call them, but it seems to be sufficient to override the buildUrl function of the store AS LONG (!) as you call the super-class 'for additional processing like adding cache-buster strings'.
* There seems to be no buildUrl override in the examplary abstract classes I am consulting, so it might be that 'this.urlpatt' is an overridable property. According to the source it is not, neither is it in the parent. 
* It appears to be that proxy need to have parameter type: 'rest', so maybe I should take a look at RestProxy-class
* Adding type: 'rest' to all store proxies.
* The buildUrl-function seems to be residing in: Ext.data.proxy.Server.buildUrl():
* "Ext JS 4.1.1a is also available under the GPLv3 for open source projects, which may require the release of your code.", counterpoint: Sencha is a member of Free Software Foundation.
* "This is a private utility class for internal use by the framework. Don't rely on its existence."
* Still no urlpatt, even in Ext.data.proxy.Server.
* [x] [?] "According to a blogger" the proxies should reside inside of a model, since the store inherits from it.

2013-01-08
* [?] Should I override pagination to ensure that all elements are visible to the user unless I wish to implement a page-select feature?
* Inquire about the autoload and autosync in stores.
* Does plugin.autodiscover() work as it should?
* Installed compass.
* The nodeadmin-application seemed not to discover the bootstrap stylesheets, while they were in order in other apps. I assumed that the reason for that was some compass routine that wasn't ran, but it was actually the template file that was changed for the subjectadmin application.
* Copied the subjectadmin's django.app template into nodeadmin. This fixes the bootstrap classes.
* The fonts are dynamically loaded.
* For each new application a folder by the name of 'resources' must be added to each static folder alongside the ext.js script folders and app.js. After the creation (and installing compass) run:
** compass install compass
* The previous point will create stylesheet, scss folders to store the scss sources and their compiled stylesheet counterparts.
* Run 'compass watch' in the resources folder to start monitorin and compiling changes.
* Use Ext.String.ellipsis on all long-names.
* Current rest-response structure does not allow me to link a child to a parent node due to the fact that the returned children are a list, and I am not sure how Ext js will handle an imposed parent property on the superlevel of each response.
* The simples solution is to create a view that contains fetches the parent reference from the NodeDetails and make it an item inside the NodeChildren-view.
* Suggest changing the role name of 'Department/Faculty/School administrator' to 'Department administrator'
* 'setPrimaryContent' (and setSecondaryContent) accept _lists_ of xtyped elements, and _do not_ accept comma-separated arguments.
* Up-navigation is fixed by adding parent to details in Ext.js: views, models and in Django: resource-fields.
* Mind the difference between 'et sted hvor', og 'et sted der'.


2012-01-09
* [x] [?] Ask for a larger dataset for testing.
* Does compass assign scope to my app, or do I have to watch out for namespace-collisions?
* Attempted to make a default Tree-view of the node hierarchy, but postponed the efforta after the deadline. The legacy code is tagged, see TreeView, TreeStore and NodeInterface classes. The API is still accessible in /devilry_nodeadmin/tree/ and served by following views.py classes: NodeTreeResource and NodeTree 
* ems or pixels?
* Some improvements in the navigation menu: the button elements look better on the right side, the key is to increase spacing between the content panes to ensure that their contents do not merge visually. 
* [x] [?] Don't forget to mention the css namespace. Maybe the sheets are condensed into one: your application may interfere with others.
* Remaining: [x] 1. move the store/view functionality into a controller. 2. introduce breadcrumbs. [x] 3. i18n and gettext. 4. Store sorting by or without statistics.. 5. fjerne default pagination
* Reading up on i18n. Probably should postpone and consult the maintainer.
* [+] Check on higher resolutions.
* [x] Translating nb->en, or en->nb?

2012-01-10
* Make font classes files: ubuntu-local and unbuntu, for the local version stored in statics and the imported one from google's font api.
* [!] Override pagination to max.
* [+] Prefix all devilry_nodeadmin styles with the name of the app; add the new style names to the components.
* [x] Fix the floating issue on the navigation links.
* [x] Capitalize view-names
* Parameters to _onRender
* For the sake of Selenium-tests you should assign cls property to all elements.
* User Datetimehelper from extjshelpers to manipulate and compare dates. Must be imported by requires. A singleton.
* Always catch errors by the callback of a store, or rather through the proxy because it has more information on hte error.

2012-01-13
* Remains: [x] 1. fix the stores, [x] 2. style bubbles as block elements; there are few, so give them some space [x] 3. translate to english and create norwegian string couterparts [x] 4. create the breadcrumbs repr in the api [x] 5. add the frontpage link, consider its position 6. disable default view pagination 7. handle pluarl/zero-count in templates
* Find out how to ignore the app-all.js in the translations.
* Improved navigation elements.
* Finished the first draft of the translation.


2012-01-17
* Fixing breadcrumbs in nodeadmin involves the fixing the _onLoad* store loading callbacks in NodeBrowserController. 
* I have completed api that returns the breadcrumb path as a sorted list of elements. This spares us for an additional http-request, error handling, and boilerplate.
* The path attribute in each node has to be translated into the breadcrumb schema: { url: "...", text: "..." }. This is done by 
* Fixed the firefox and IE issue: js attempts to parse html-output of rest framework instead of the json. Seems to be a url-issue, which requires and appendage of format=json to the url.
* Fixed the breadcrumb with a temporary solution that does what it should. Some tuning is due to display the current node (the last element in the path that doesn't have a link).
* Complete nodeLookup.js for nodeadmin, use in breadcrumbs.
* [+] Disable default view pagination.
* [+] Deal with pluralization and zero element count.
* [+] Store sorting
* [x] Fix the date format in node browser.
* To use the facilities of devilry_extjsextras add the custom template function to your template, after the last comma. 
** {
**     formatDatetime: function(datetime) {
**       return devilry_extjsextras.DatetimeHelpers.formatDateTimeShort(
**           datetime)
**       }
** }
* If you have problems in accessing certain parameters in a template, and passing them to a template-function: try prepending the parameter name with a "values.", like:
** this.formatDatetime( values.most_recent_start_time )
* [+] Find out how to exclude app-all.js
* Something happened to the related top-level nodes.
* Fixed the stores and correct import in the 
* The built code seems to be interfering.