forked from t11e/discovery_datatool
-
Notifications
You must be signed in to change notification settings - Fork 1
/
README
166 lines (124 loc) · 6.86 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
The Discovery Data Tool provides a cross-platform implementation of a Changeset
Publisher. It uses JDBC to connect to the database and is thus compatible
with most major databases.
Documentation can be found at:
https://transparensee.com/docs
To build the release run:
ant clean release
To use the release, get the built zip from
build.ant/discovery_datatool.zip
Unpack it and create a discovery_datatool.xml configuration file. The release
includes example configuration files in the examples directory of the zip file.
Then run the tool by executing the run.sh script.
Release Notes
-------------
1.12 - 2012-08-10
* Updates writing of changeset XML to all item properties to be empty.
Previously, a null value would cause a runtime exception while generating
the cahngeset, and the resulting changeset XML would be invalid.
1.11 - 2012-01-16
* Adds validation of changeset SQL against the database on startup. May be
skipped by setting validate_sql=false in datatool.properties.
* Adds progress logging of changeset creation. Uses same progress log message
formatting as Discovery Engine.
* Makes all references to columns in XML configuration attributes
case-insensitive (e.g. idColumn, provider, kind, discriminator,
jsonColumns, jsonColumnNames, scopedJsonColumnNames, mergeColumns/@name and
mergeColumns/@value).
* Updates discovery_datatool shell script to default the http port to 8089.
Note that if you want to listen for both HTTP and HTTPS, then you need to
specify both http_port and https_port in datatool.properties.
* Removes extraneous chatter and warnings from log output.
1.10 - 2012-01-03
* Simplifies process of setting up init.d script. Startup options are largely
specified in datatool.properties. See init-script/README for detailed
instructions. Removes use of jakarta-commons-daemon (jsvc).
* Changes logging.properties to log to a file with rotation at 10mb keeping
up to 30 files.
1.9 - 2011-12-26
* Fixes defect in add-to-item configuration parsing: add-to-item was only
read from delta configuration elements. add-to-item elements in snapshots
were ignored. delta add-to-item elements were incorrectly used by the
publisher's snapshots.
1.8 - 2011-12-08
* Adds configuration attribute for item property-case (preserve, lower, upper
and legacy). Starting with new schema version 7, the default is preserve. Older
schema versions continue to have the legacy behavior of lowercasing
property names from column names but not from pivoted values.
preserve - Use whatever case we get back from the database
lower - All property names are converted to locale lower case
upper - All property names are converted to locale upper case
legacy - Property names from sql column labels are converted to lower case,
but property names from vertical table values are preserved.
* Fixes "If a subquery is empty, it should be ignored (and reported) and not
throw an error."
* Adds support for unscopedJsonColumns to be able to merge in JSON column
data to the top level properties. Deprecates property "jsonColumnNames" in
favor of more explicit "scopedJsonColumns" for the same behavior. Adds new
property "unscopedJsonColumns" for columns that should be merged into the
top-level item properties.
* Makes subqueries use NullDefaultCaseInsensitiveParameterSource. If subquery
uses a bound parameter that is not present in the main item properties, then
null is used for the parameter value.
1.7 - 2011-11-04
* Adds support for add-to-item for deltas and snapshots (the Discovery Engine
does not support add-to-item for bulk or full changesets). The add-to-item
configuration element supports the same attributes and child configuration
elements as set-item.
* Adds another subquery test scenario.
* Fixes Data Tool Outputs Empty Strings: If a column returns an empty string,
the data tool still outputs the column value. Empty strings should not be
output.
* Fixes merge-columns should not be case sensitive.
1.6 - 2011-09-28
* Adds support for provider and kind to Discovery Data Tool. The
configuration elements can now take optional attributes providerColumn and
kindColumn. If either of these are specified, then the resulting changeset
set-item and remove-item elements will not have an id attribute, and instead
will have locator, provider and kind attributes. The providerColumn and
kindColumn are not included in the resulting set-item properties, since that
would be redundant.
* No longer includes id column in set-item properties, since it is redundant
to the set-item id attribute.
1.5 - 2011-08-04
* Adds support for "vertical" tables.
1.4 - 2011-03-08
* Creates new version of Data Tool Configuration XML schema to support bulk
and full changesets and subqueries.
* Adds support for bulk and full changesets in addition to snapshots.
* Adds support for a wide variety of subquery types in order to better
support flattening complex database structures without the use of views or
stored procedures.
* Fixes WebServerMain to handle paths with spaces in directory names. Fixes
problem where datatool cannot be run if its path contains spaces.
* Updates tests to verify tool continutes to be backwards compatible in
parsing all versions of the configuration xml.
* Adds support for column aliases.
* Makes named SQL parameter matching case insensitive by creating and using
CaseInsensitveParameterSource (based on Spring's MapSqlParameterSource,
which unfortunately does not allow plugging in a different Map
implementation). All tests now pass.
* Updates logging.properties to log com.t11e.discovery.datatool at FINE. And
adds a comment explaining how to enable logging sql query execution speed.
Turns down default level to WARNING (which quiets Spring).
* Adds test to show that Discovery Data Tool can call stored procedures.
1.3 - 2010-10-01
* Adds forceSnapshot boolean parameter to ChangesetController. If true, then
start date of null is used and a snapshot is created regardless of the
profile's last run time.
1.2 - 2010-09-07
* Adds support for protecting incoming connections via SSL and basic
authentication.
* Adds throwing an exception in config XML does not have a root config
element in a recognized namespace.
1.1 - 2010-06-11
* Added createSql support to sqlProfile, used to auto create missing
changeset profiles.
* Bumped the schema from http://transparensee.com/schema/datatool-config-1
to http://transparensee.com/schema/datatool-config-2.
* Added init script to the built release, depends upon Apache Commons Daemon
jsvc (http://commons.apache.org/daemon/jsvc.html).
* Fixed Configuration parsing issue where long text elements in the XML
were being truncated.
1.0 - 2010-06-03
* Initial release