forked from apache/airflow
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathCHANGELOG.txt
715 lines (703 loc) · 36.1 KB
/
CHANGELOG.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
<<<<<<< HEAD
AIRFLOW 1.8.1, 2017-05-09
-------------------------
[AIRFLOW-1142] SubDAG Tasks Not Executed Even Though All Dependencies Met
[AIRFLOW-1138] Add licenses to files in scripts directory
[AIRFLOW-1127] Move license notices to LICENSE instead of NOTICE
[AIRFLOW-1124] Do not set all task instances to scheduled on backfill
[AIRFLOW-1120] Update version view to include Apache prefix
[AIRFLOW-1062] DagRun#find returns wrong result if external_trigger=False is specified
[AIRFLOW-1054] Fix broken import on test_dag
[AIRFLOW-1050] Retries ignored - regression
[AIRFLOW-1033] TypeError: can't compare datetime.datetime to None
[AIRFLOW-1017] get_task_instance should return None instead of throw an exception for non-existent TIs
[AIRFLOW-1011] Fix bug in BackfillJob._execute() for SubDAGs
[AIRFLOW-1004] `airflow webserver -D` runs in foreground
[AIRFLOW-1001] Landing Time shows "unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'" on example_subdag_operator
[AIRFLOW-1000] Rebrand to Apache Airflow instead of Airflow
[AIRFLOW-989] Clear Task Regression
[AIRFLOW-974] airflow.util.file mkdir has a race condition
[AIRFLOW-906] Update Code icon from lightning bolt to file
[AIRFLOW-858] Configurable database name for DB operators
[AIRFLOW-853] ssh_execute_operator.py stdout decode default to ASCII
[AIRFLOW-832] Fix debug server
[AIRFLOW-817] Trigger dag fails when using CLI + API
[AIRFLOW-816] Make sure to pull nvd3 from local resources
[AIRFLOW-815] Add previous/next execution dates to available default variables.
[AIRFLOW-813] Fix unterminated unit tests in tests.job (tests/job.py)
[AIRFLOW-812] Scheduler job terminates when there is no dag file
[AIRFLOW-806] UI should properly ignore DAG doc when it is None
[AIRFLOW-794] Consistent access to DAGS_FOLDER and SQL_ALCHEMY_CONN
[AIRFLOW-785] ImportError if cgroupspy is not installed
[AIRFLOW-784] Cannot install with funcsigs > 1.0.0
[AIRFLOW-780] The UI no longer shows broken DAGs
[AIRFLOW-777] dag_is_running is initlialized to True instead of False
[AIRFLOW-719] Skipped operations make DAG finish prematurely
[AIRFLOW-694] Empty env vars do not overwrite non-empty config values
[AIRFLOW-492] Insert into dag_stats table results into failed task while task itself succeeded
[AIRFLOW-139] Executing VACUUM with PostgresOperator
[AIRFLOW-111] DAG concurrency is not honored
[AIRFLOW-88] Improve clarity Travis CI reports
=======
AIRFLOW 1.8.2rc1, 2017-06-05
-------------------------
9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing Booleans
333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
93825d5 Re-enable caching for hadoop components
33a9dcb Pin Hive and Hadoop to a specific version and create writable warehouse dir
7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
c6a09c4 Updating CHANGELOG for 1.8.2rc1
570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF requirements
e10af9a [AIRFLOW-XXX] Set version to 1.8.2rc1
69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2 templates
6de5330 [AIRFLOW-1119] Fix unload query so headers are on first row[]
b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency limits
a2c18a5 [AIRFLOW-1064] Change default sort to job_id for TaskInstanceModelView
d1c64ab [AIRFLOW-1038] Specify celery serialization options explicitly
b4ee88a [AIRFLOW-1036] Randomize exponential backoff
9fca409 [AIRFLOW-993] Update date inference logic
272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook modification time
c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
acd0166 [AIRFLOW-1263] Dynamic height for charts
7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
2e9eee3 [AIRFLOW-1282] Fix known event column sorting
2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
57d5bcd [AIRFLOW-1281] Sort variables by key field by default
802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
51a83cc [AIRFLOW-1199] Fix create modal
cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty key
5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by execution_date
f446c08 [AIRFLOW-1145] Fix closest_date_partition function with before set to True If we're looking for the closest date before, we should take the latest date in the list of date before.
93b8e96 [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately
093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[]
777f181 [AIRFLOW-1168] Add closing() to all connections and cursors
bc8e912 [AIRFLOW-XXX] Updating CHANGELOG, README, and UPDATING after 1.8.1 release
AIRFLOW 1.8.1, 2017-05-09
-------------------------
[AIRFLOW-1160] Update Spark parameters for Mesos
[AIRFLOW 1149] Allow for custom filters in Jinja2 templates
[AIRFLOW-1119] Fix unload query so headers are on first row[]
[AIRFLOW-1089] Add Spark application arguments
[AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
[AIRFLOW-1074] Don't count queued tasks for concurrency limits
[AIRFLOW-1064] Change default sort to job_id for TaskInstanceModelView
[AIRFLOW-1038] Specify celery serialization options explicitly
[AIRFLOW-1036] Randomize exponential backoff
[AIRFLOW-993] Update date inference logic
[AIRFLOW-1167] Support microseconds in FTPHook modification time
[AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
[AIRFLOW-1263] Dynamic height for charts
[AIRFLOW-1266] Increase width of gantt y axis
[AIRFLOW-1290] set docs author to 'Apache Airflow'
[AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
[AIRFLOW-1192] Some enhancements to qubole_operator
[AIRFLOW-1281] Sort variables by key field by default
[AIRFLOW-1244] Forbid creation of a pool with empty name
[AIRFLOW-1243] DAGs table has no default entries to show
[AIRFLOW-1227] Remove empty column on the Logs view
[AIRFLOW-1226] Remove empty column on the Jobs view
[AIRFLOW-1199] Fix create modal
[AIRFLOW-1200] Forbid creation of a variable with an empty key
[AIRFLOW-1186] Sort dag.get_task_instances by execution_date
[AIRFLOW-1145] Fix closest_date_partition function with before set to True If we're looking for the closest date before, we should take the latest date in the list of date before.
[AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
[AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately
[AIRFLOW-1150] Fix scripts execution in sparksql hook[]
[AIRFLOW-1168] Add closing() to all connections and cursors
>>>>>>> Stashed changes
>>>>>>> 1.8.2+activate_virtualenv
AIRFLOW 1.8.0, 2017-03-12
-------------------------
[AIRFLOW-900] Double trigger should not kill original task instance
[AIRFLOW-900] Fixes bugs in LocalTaskJob for double run protection
[AIRFLOW-932] Do not mark tasks removed when backfilling
[AIRFLOW-961] run onkill when SIGTERMed
[AIRFLOW-910] Use parallel task execution for backfills
[AIRFLOW-967] Wrap strings in native for py2 ldap compatibility
[AIRFLOW-941] Use defined parameters for psycopg2
[AIRFLOW-719] Prevent DAGs from ending prematurely
[AIRFLOW-938] Use test for True in task_stats queries
[AIRFLOW-937] Improve performance of task_stats
[AIRFLOW-933] use ast.literal_eval rather eval because ast.literal_eval does not execute input.
[AIRFLOW-925] Revert airflow.hooks change that cherry-pick picked
[AIRFLOW-919] Running tasks with no start date shouldn't break a DAGs UI
[AIRFLOW-802] Add spark-submit operator/hook
[AIRFLOW-897] Prevent dagruns from failing with unfinished tasks
[AIRFLOW-861] make pickle_info endpoint be login_required
[AIRFLOW-853] use utf8 encoding for stdout line decode
[AIRFLOW-856] Make sure execution date is set for local client
[AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log verbosity
[AIRFLOW-831] Restore import to fix broken tests
[AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN exclusively from settings
[AIRFLOW-694] Fix config behaviour for empty envvar
[AIRFLOW-365] Set dag.fileloc explicitly and use for Code view
[AIRFLOW-931] Do not set QUEUED in TaskInstances
[AIRFLOW-899] Tasks in SCHEDULED state should be white in the UI instead of black
[AIRFLOW-895] Address Apache release incompliancies
[AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a dagrun has no start date
[AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer
[AIRFLOW-863] Example DAGs should have recent start dates
[AIRFLOW-869] Refactor mark success functionality
[AIRFLOW-856] Make sure execution date is set for local client
[AIRFLOW-814] Fix Presto*CheckOperator.__init__
[AIRFLOW-844] Fix cgroups directory creation
[AIRFLOW-816] Use static nvd3 and d3
[AIRFLOW-821] Fix py3 compatibility
[AIRFLOW-817] Check for None value of execution_date in endpoint
[AIRFLOW-822] Close db before exception
[AIRFLOW-815] Add prev/next execution dates to template variables
[AIRFLOW-813] Fix unterminated unit tests in SchedulerJobTest
[AIRFLOW-813] Fix unterminated scheduler unit tests
[AIRFLOW-806] UI should properly ignore DAG doc when it is None
[AIRFLOW-812] Fix the scheduler termination bug.
[AIRFLOW-780] Fix dag import errors no longer working
[AIRFLOW-783] Fix py3 incompatibility in BaseTaskRunner
[AIRFLOW-810] Correct down_revision dag_id/state index creation
[AIRFLOW-807] Improve scheduler performance for large DAGs
[AIRFLOW-798] Check return_code before forcing termination
[AIRFLOW-139] Let psycopg2 handle autocommit for PostgresHook
[AIRFLOW-776] Add missing cgroups devel dependency
[AIRFLOW-777] Fix expression to check if a DagRun is in running state
[AIRFLOW-785] Don't import CgroupTaskRunner at global scope
[AIRFLOW-784] Pin funcsigs to 1.0.0
[AIRFLOW-624] Fix setup.py to not import airflow.version as version
[AIRFLOW-779] Task should fail with specific message when deleted
[AIRFLOW-778] Fix completey broken MetastorePartitionSensor
[AIRFLOW-739] Set pickle_info log to debug
[AIRFLOW-771] Make S3 logs append instead of clobber
[AIRFLOW-773] Fix flaky datetime addition in api test
[AIRFLOW-219][AIRFLOW-398] Cgroups + impersonation
[AIRFLOW-683] Add jira hook, operator and sensor
[AIRFLOW-762] Add Google DataProc delete operator
[AIRFLOW-760] Update systemd config
[AIRFLOW-759] Use previous dag_run to verify depend_on_past
[AIRFLOW-757] Set child_process_log_directory default more sensible
[AIRFLOW-692] Open XCom page to super-admins only
[AIRFLOW-737] Fix HDFS Sensor directory.
[AIRFLOW-747] Fix retry_delay not honoured
[AIRFLOW-558] Add Support for dag.catchup=(True|False) Option
[AIRFLOW-489] Allow specifying execution date in trigger_dag API
[AIRFLOW-738] Commit deleted xcom items before insert
[AIRFLOW-729] Add Google Cloud Dataproc cluster creation operator
[AIRFLOW-728] Add Google BigQuery table sensor
[AIRFLOW-741] Log to debug instead of info for app.py
[AIRFLOW-731] Fix period bug for NamedHivePartitionSensor
[AIRFLOW-740] Pin jinja2 to < 2.9.0
[AIRFLOW-663] Improve time units for task performance charts
[AIRFLOW-665] Fix email attachments
[AIRFLOW-734] Fix SMTP auth regression when not using user/pass
[AIRFLOW-702] Fix LDAP Regex Bug
[AIRFLOW-717] Add Cloud Storage updated sensor
[AIRFLOW-695] Retries do not execute because dagrun is in FAILED state
[AIRFLOW-673] Add operational metrics test for SchedulerJob
[AIRFLOW-727] try_number is not increased
[AIRFLOW-715] A more efficient HDFS Sensor:
[AIRFLOW-716] Allow AVRO BigQuery load-job without schema
[AIRFLOW-718] Allow the query URI for DataProc Pig
Log needs to be part of try/catch block
[AIRFLOW-721] Descendant process can disappear before termination
[AIRFLOW-403] Bash operator's kill method leaves underlying processes running
[AIRFLOW-657] Add AutoCommit Parameter for MSSQL
[AIRFLOW-641] Improve pull request instructions
[AIRFLOW-685] Add test for MySqlHook.bulk_load()
[AIRFLOW-686] Match auth backend config section
[AIRFLOW-691] Add SSH KeepAlive option to SSH_hook
[AIRFLOW-709] Use same engine for migrations and reflection
[AIRFLOW-700] Update to reference to web authentication documentation
[AIRFLOW-649] Support non-sched DAGs in LatestOnlyOp
[AIRFLOW-712] Fix AIRFLOW-667 to use proper HTTP error properties
[AIRFLOW-710] Add OneFineStay as official user
[AIRFLOW-703][AIRFLOW-1] Stop Xcom being cleared too early
[AIRFLOW-679] Stop concurrent task instances from running
[AIRFLOW-704][AIRFLOW-1] Fix invalid syntax in BQ hook
[AIRFLOW-667] Handle BigQuery 503 error
[AIRFLOW-680] Disable connection pool for commands
[AIRFLOW-678] Prevent scheduler from double triggering TIs
[AIRFLOW-677] Kill task if it fails to heartbeat
[AIRFLOW-674] Ability to add descriptions for DAGs
[AIRFLOW-682] Bump MAX_PERIODS to make mark_success work for large DAGs
Use jdk selector to set required jdk
[AIRFLOW-647] Restore dag.get_active_runs
[AIRFLOW-662] Change seasons to months in project description
[AIRFLOW-656] Add dag/task/date index to xcom table
[AIRFLOW-658] Improve schema_update_options in GCP
[AIRFLOW-41] Fix pool oversubscription
[AIRFLOW-489] Add API Framework
[AIRFLOW-653] Add some missing endpoint tests
[AIRFLOW-652] Remove obsolete endpoint
[AIRFLOW-345] Add contrib ECSOperator
[AIRFLOW-650] Adding Celect to user list
[AIRFLOW-510] Filter Paused Dags, show Last Run & Trigger Dag
[AIRFLOW-643] Improve date handling for sf_hook
[AIRFLOW-638] Add schema_update_options to GCP ops
[AIRFLOW-640] Install and enable nose-ignore-docstring
[AIRFLOW-639]AIRFLOW-639] Alphasort package names
[AIRFLOW-375] Fix pylint errors
[AIRFLOW-347] Show empty DAG runs in tree view
[AIRFLOW-628] Adding SalesforceHook to contrib/hooks
[AIRFLOW-514] hive hook loads data from pandas DataFrame into hive and infers types
[AIRFLOW-565] Fixes DockerOperator on Python3.x
[AIRFLOW-635] Encryption option for S3 hook
[AIRFLOW-137] Fix max_active_runs on clearing tasks
[AIRFLOW-343] Fix schema plumbing in HiveServer2Hook
[AIRFLOW-130] Fix ssh operator macosx
[AIRFLOW-633] Show TI attributes in TI view
[AIRFLOW-626][AIRFLOW-1] HTML Content does not show up when sending email with attachment
[AIRFLOW-533] Set autocommit via set_autocommit
[AIRFLOW-629] stop pinning lxml
[AIRFLOW-464] Add setdefault method to Variable
[AIRFLOW-626][AIRFLOW-1] HTML Content does not show up when sending email with attachment
[AIRFLOW-591] Add datadog hook & sensor
[AIRFLOW-561] Add RedshiftToS3Transfer operator
[AIRFLOW-570] Pass root to date form on gantt
[AIRFLOW-504] Store fractional seconds in MySQL tables
[AIRFLOW-623] LDAP attributes not always a list
[AIRFLOW-611] source_format in BigQueryBaseCursor
[AIRFLOW-619] Fix exception in Gannt chart
[AIRFLOW-618] Cast DateTimes to avoid sqllite errors
[AIRFLOW-422] Add JSON endpoint for task info
[AIRFLOW-616][AIRFLOW-617] Minor fixes to PR tool UX
[AIRFLOW-179] Fix DbApiHook with non-ASCII chars
[AIRFLOW-566] Add timeout while fetching logs
[AIRFLOW-615] Set graph glyphicon first
[AIRFLOW-609] Add application_name to PostgresHook
[AIRFLOW-604] Revert .first() to .one()
[AIRFLOW-370] Create AirflowConfigException in exceptions.py
[AIRFLOW-582] Fixes TI.get_dagrun filter (removes start_date)
[AIRFLOW-568] Fix double task_stats count if a DagRun is active
[AIRFLOW-585] Fix race condition in backfill execution loop
[AIRFLOW-580] Prevent landscape warning on .format
[AIRFLOW-597] Check if content is None, not false-equivalent
[AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m.
[AIRFLOW-453] Add XCom Admin Page
[AIRFLOW-588] Add Google Cloud Storage Object sensor[]
[AIRFLOW-592] example_xcom import Error
[AIRFLOW-587] Fix incorrect scope for Google Auth[]
[AIRFLOW-589] Add templatable job_name[]
[AIRFLOW-227] Show running config in config view
[AIRFLOW-319]AIRFLOW-319] xcom push response in HTTP Operator
[AIRFLOW-385] Add symlink to latest scheduler log directory
[AIRFLOW-583] Fix decode error in gcs_to_bq
[AIRFLOW-96] s3_conn_id using environment variable
[AIRFLOW-575] Clarify tutorial and FAQ about `schedule_interval` always inheriting from DAG object
[AIRFLOW-577] Output BigQuery job for improved debugging
[AIRFLOW-560] Get URI & SQLA engine from Connection
[AIRFLOW-518] Require DataProfilingMixin for Variables CRUD
[AIRFLOW-553] Fix load path for filters.js
[AIRFLOW-554] Add Jinja support to Spark-sql
[AIRFLOW-550] Make ssl config check empty string safe
[AIRFLOW-500] Use id for github allowed teams
[AIRFLOW-556] Add UI PR guidelines
[AIRFLOW-358][AIRFLOW-430] Add `connections` cli
[AIRFLOW-548] Load DAGs immediately & continually
[AIRFLOW-539] Updated BQ hook and BQ operator to support Standard SQL.
[AIRFLOW-378] Add string casting to params of spark-sql operator
[AIRFLOW-544] Add Pause/Resume toggle button
[AIRFLOW-333][AIRFLOW-258] Fix non-module plugin components
[AIRFLOW-542] Add tooltip to DAGs links icons
[AIRFLOW-530] Update docs to reflect connection environment var has to be in uppercase
[AIRFLOW-525] Update template_fields in Qubole Op
[AIRFLOW-480] Support binary file download from GCS
[AIRFLOW-198] Implement latest_only_operator
[AIRFLOW-91] Add SSL config option for the webserver
[AIRFLOW-191] Fix connection leak with PostgreSQL backend
[AIRFLOW-512] Fix 'bellow' typo in docs & comments
[AIRFLOW-509][AIRFLOW-1] Create operator to delete tables in BigQuery
[AIRFLOW-498] Remove hard-coded gcp project id
[AIRFLOW-505] Support unicode characters in authors' names
[AIRFLOW-494] Add per-operator success/failure metrics
[AIRFLOW-488] Fix test_simple fail
[AIRFLOW-468] Update Panda requirement to 0.17.1
[AIRFLOW-159] Add cloud integration section + GCP documentation
[AIRFLOW-477][AIRFLOW-478] Restructure security section for clarity
[AIRFLOW-467] Allow defining of project_id in BigQueryHook
[AIRFLOW-483] Change print to logging statement
[AIRFLOW-475] make the segment granularity in Druid hook configurable
AIRFLOW 1.7.2
-------------
[AIRFLOW-463] Link Airflow icon to landing page
[AIRFLOW-149] Task Dependency Engine + Why Isn't My Task Running View
[AIRFLOW-361] Add default failure handler for the Qubole Operator
[AIRFLOW-353] Fix dag run status update failure
[AIRFLOW-447] Store source URIs in Python 3 compatible list
[AIRFLOW-443] Make module names unique when importing
[AIRFLOW-444] Add Google authentication backend
[AIRFLOW-446][AIRFLOW-445] Adds missing dataproc submit options
[AIRFLOW-431] Add CLI for CRUD operations on pools
[AIRFLOW-329] Update Dag Overview Page with Better Status Columns
[AIRFLOW-360] Fix style warnings in models.py
[AIRFLOW-425] Add white fill for null state tasks in tree view.
[AIRFLOW-69] Use dag runs in backfill jobs
[AIRFLOW-415] Make dag_id not found error clearer
[AIRFLOW-416] Use ordinals in README's company list
[AIRFLOW-369] Allow setting default DAG orientation
[AIRFLOW-410] Add 2 Q/A to the FAQ in the docs
[AIRFLOW-407] Add different colors for some sensors
[AIRFLOW-414] Improve error message for missing FERNET_KEY
[AIRFLOW-406] Sphinx/rst fixes
[AIRFLOW-412] Fix lxml dependency
[AIRFLOW-413] Fix unset path bug when backfilling via pickle
[AIRFLOW-78] Airflow clear leaves dag_runs
[AIRFLOW-402] Remove NamedHivePartitionSensor static check, add docs
[AIRFLOW-394] Add an option to the Task Duration graph to show cumulative times
[AIRFLOW-404] Retry download if unpacking fails for hive
[AIRFLOW-276] Gunicorn rolling restart
[AIRFLOW-399] Remove dags/testdruid.py
[AIRFLOW-400] models.py/DAG.set_dag_runs_state() does not correctly set state
[AIRFLOW-395] Fix colon/equal signs typo for resources in default config
[AIRFLOW-397] Documentation: Fix typo "instatiating" to "instantiating"
[AIRFLOW-395] Remove trailing commas from resources in config
[AIRFLOW-388] Add a new chart for Task_Tries for each DAG
[AIRFLOW-322] Fix typo in FAQ section
[AIRFLOW-375] Pylint fixes
limit scope to user email only AIRFLOW-386
[AIRFLOW-383] Cleanup example qubole operator dag
[AIRFLOW-160] Parse DAG files through child processes
[AIRFLOW-381] Manual UI Dag Run creation: require dag_id field
[AIRFLOW-373] Enhance CLI variables functionality
[AIRFLOW-379] Enhance Variables page functionality: import/export variables
[AIRFLOW-331] modify the LDAP authentication config lines in 'Security' sample codes
[AIRFLOW-356][AIRFLOW-355][AIRFLOW-354] Replace nobr, enable DAG only exists locally message, change edit DAG icon
[AIRFLOW-362] Import __future__ division
[AIRFLOW-359] Pin flask-login to 0.2.11
[AIRFLOW-261] Add bcc and cc fields to EmailOperator
[AIRFLOW-348] Fix code style warnings
[AIRFLOW-349] Add metric for number of zombies killed
[AIRFLOW-340] Remove unused dependency on Babel
[AIRFLOW-339]: Ability to pass a flower conf file
[AIRFLOW-341][operators] Add resource requirement attributes to operators
[AIRFLOW-335] Fix simple style errors/warnings
[AIRFLOW-337] Add __repr__ to VariableAccessor and VariableJsonAccessor
[AIRFLOW-334] Fix using undefined variable
[AIRFLOW-315] Fix blank lines code style warnings
[AIRFLOW-306] Add Spark-sql Hook and Operator
[AIRFLOW-327] Add rename method to the FTPHook
[AIRFLOW-321] Fix a wrong code example about tests/dags
[AIRFLOW-316] Always check DB state for Backfill Job execution
[AIRFLOW-264] Adding workload management for Hive
[AIRFLOW-297] support exponential backoff option for retry delay
[AIRFLOW-31][AIRFLOW-200] Add note to updating.md
[AIRFLOW-307] There is no __neq__ python magic method.
[AIRFLOW-309] Add requirements of develop dependencies to docs
[AIRFLOW-307] Rename __neq__ to __ne__ python magic method.
[AIRFLOW-313] Fix code style for sqoop_hook.py
[AIRFLOW-311] Fix wrong path in CONTRIBUTING.md
[AIRFLOW-24] DataFlow Java Operator
[AIRFLOW-308] Add link to refresh DAG within DAG view header
[AIRFLOW-314] Fix BigQuery cursor run_table_upsert method
[AIRFLOW-298] fix incubator diclaimer in docs
[AIRFLOW-284] HiveServer2Hook fix for cursor scope for get_results
[AIRFLOW-260] More graceful exit when issues can't be closed
[AIRFLOW-260] Handle case when no version is found
[AIRFLOW-228] Handle empty version list in PR tool
[AIRFLOW-302] Improve default squash commit message
[AIRFLOW-187] Improve prompt styling
[AIRFLOW-187] Fix typo in argument name
[AIRFLOW-187] Move "Close XXX" message to end of squash commit
[AIRFLOW-247] Add EMR hook, operators and sensors. Add AWS base hook
[AIRFLOW-301] Fix broken unit test
[AIRFLOW-100] Add execution_date_fn to ExternalTaskSensor
[AIRFLOW-282] Remove PR Tool logic that depends on version formatting
[AIRFLOW-291] Add index for state in TI table
[AIRFLOW-269] Add some unit tests for PostgreSQL
[AIRFLOW-296] template_ext is being treated as a string rather than a tuple in qubole operator
[AIRFLOW-286] Improve FTPHook to implement context manager interface
[AIRFLOW-243] Create NamedHivePartitionSensor
[AIRFLOW-246] Improve dag_stats endpoint query
[AIRFLOW-189] Highlighting of Parent/Child nodes in Graphs
[ARFLOW-255] Check dagrun timeout when comparing active runs
[AIRFLOW-281] Add port to mssql_hook
[AIRFLOW-285] Use Airflow 2.0 style imports for all remaining hooks/operators
[AIRFLOW-40] Add LDAP group filtering feature.
[AIRFLOW-277] Multiple deletions does not work in Task Instances view if using SQLite backend
[AIRFLOW-200] Make hook/operator imports lazy, and print proper exceptions
[AIRFLOW-283] Make store_to_xcom_key a templated field in GoogleCloudStorageDownloadOperator
[AIRFLOW-278] Support utf-8 ecoding for SQL
[AIRFLOW-280] clean up tmp druid table no matter if an ingestion job succeeds or not
[AIRFLOW-274] Add XCom functionality to GoogleCloudStorageDownloadOperator
[AIRFLOW-273] Create an svg version of the airflow logo.
[AIRFLOW-275] Update contributing guidelines
[AIRFLOW-244] Modify hive operator to inject analysis data
[AIRFLOW-162] Allow variable to be accessible into templates
[AIRFLOW-248] Add Apache license header to all files
[AIRFLOW-263] Remove temp backtick file
[AIRFLOW-252] Raise Sqlite exceptions when deleting tasks instance in WebUI
[AIRFLOW-180] Fix timeout behavior for sensors
[AIRFLOW-262] Simplify commands in MANIFEST.in
[AIRFLOW-31] Add zope dependency
[AIRFLOW-6] Remove dependency on Highcharts
[AIRFLOW-234] make task that aren't `running` self-terminate
[AIRFLOW-256] Fix test_scheduler_reschedule heartrate
Add Python 3 compatibility fix
[AIRFLOW-31] Use standard imports for hooks/operators
[AIRFLOW-173] Initial implementation of FileSensor
[AIRFLOW-224] Collect orphaned tasks and reschedule them
[AIRFLOW-239] Fix tests indentation
[AIRFLOW-225] Better units for task duration graph
[AIRFLOW-241] Add testing done section to PR template
[AIRFLOW-222] Show duration of task instances in ui
[AIRFLOW-231] Do not eval user input in PrestoHook
[AIRFLOW-216] Add Sqoop Hook and Operator
[AIRFLOW-171] Add upgrade notes on email and S3 to 1.7.1.2
[AIRFLOW-238] Make compatible with flask-admin 1.4.1
[AIRFLOW-230] [HiveServer2Hook] adding multi statements support
[AIRFLOW-142] setup_env.sh doesn't download hive tarball if hdp is specified as distro
[AIRFLOW-223] Make parametrable the IP on which Flower binds to
[AIRFLOW-218] Added option to enable webserver gunicorn access/err logs
[AIRFLOW-213] Add "Closes #X" phrase to commit messages
[AIRFLOW-68] Align start_date with the schedule_interval
[AIRFLOW-9] Improving docs to meet Apache's standards
[AIRFLOW-131] Make XCom.clear more selective
[AIRFLOW-214] Fix occasion of detached taskinstance
[AIRFLOW-206] Add commit to close PR
[AIRFLOW-206] Always load local log files if they exist
[AIRFLOW-211] Fix JIRA "resolve" vs "close" behavior
[AIRFLOW-64] Add note about relative DAGS_FOLDER
[AIRFLOW-114] Sort plugins dropdown
[AIRFLOW-209] Add scheduler tests and improve lineage handling
[AIRFLOW-207] Improve JIRA auth workflow
[AIRFLOW-187] Improve PR tool UX
[AIRFLOW-155] Documentation of Qubole Operator
Optimize and refactor process_dag
[AIRFLOW-185] Handle empty versions list
[AIRFLOW-201] Fix for HiveMetastoreHook + kerberos
[AIRFLOW-202]: Fixes stray print line
[AIRFLOW-196] Fix bug that exception is not handled in HttpSensor
[AIRFLOW-195] : Add toggle support to subdag clearing in the CLI
[AIRFLOW-23] Support for Google Cloud DataProc
[AIRFLOW-25] Configuration for Celery always required
[AIRFLOW-190] Add codecov and remove download count
[AIRFLOW-168] Correct evaluation of @once schedule
[AIRFLOW-183] Fetch log from remote when worker returns 4xx/5xx response
[AIRFLOW-181] Fix failing unpacking of hadoop by redownloading
[AIRFLOW-176] remove unused formatting key
[AIRFLOW-167]: Add dag_state option in cli
[AIRFLOW-178] Fix bug so that zip file is detected in DAG folder
[AIRFLOW-176] Improve PR Tool JIRA workflow
AIRFLOW-45: Support Hidden Airflow Variables
[AIRFLOW-175] Run git-reset before checkout in PR tool
[AIRFLOW-157] Make PR tool Py3-compat; add JIRA command
[AIRFLOW-170] Add missing @apply_defaults
AIRFLOW 1.7.1, 2016-05-19
-------------------------
- Fix : Don't treat premature tasks as could_not_run tasks
- AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/incubator-airflow#1485
- Add logic to lock DB and avoid race condition
- Handle queued tasks from multiple jobs/executors
- AIRFLOW-52 Warn about overwriting tasks in a DAG
- Fix corner case with joining processes/queues (#1473)
- [AIRFLOW-52] Fix bottlenecks when working with many tasks
- Add columns to toggle extra detail in the connection list view.
- Log the number of errors when importing DAGs
- Log dagbag metrics dupplicate messages in queue into Statsd (#1406)
- Clean up issue template (#1419)
- correct missed arg.foreground to arg.daemon in cli
- Reinstate imports for github enterprise auth
- Use os.execvp instead of subprocess.Popen for the webserver
- Revert from using "--foreground" to "--daemon"
- Implement a Cloudant hook
- Add missing args to `airflow clear`
- Fixed a bug in the scheduler: num_runs used where runs intended
- Add multiprocessing support to the scheduler
- Partial fix to make sure next_run_date cannot be None
- Support list/get/set variables in the CLI
- Properly handle BigQuery booleans in BigQuery hook.
- Added the ability to view XCom variables in webserver
- Change DAG.tasks from a list to a dict
- Add support for zipped dags
- Stop creating hook on instantiating of S3 operator
- User subquery in views to find running DAGs
- Prevent DAGs from being reloaded on every scheduler iteration
- Add a missing word to docs
- Document the parameters of `DbApiHook`
- added oracle operator with existing oracle hook
- Add PyOpenSSL to Google cloud gcp_api.
- Remove executor error unit test
- Add DAG inference, deferral, and context manager
- Don't return error when writing files to Google cloud storage.
- Fix GCS logging for gcp_api.
- Ensure attr is in scope for error message
- Fixing misnamed PULL_REQUEST_TEMPLATE
- Extract non_pooled_task_slot_count into a configuration param
- Update plugins.rst for clarity on the example (#1309)
- Fix s3 logging issue
- Add twitter feed example dag
- Github ISSUE_TEMPLATE & PR_TEMPLATE cleanup
- Reduce logger verbosity
- Adding a PR Template
- Add Lucid to list of users
- Fix usage of asciiart
- Use session instead of outdated main_session for are_dependencies_met
- Fix celery flower port allocation
- Fix for missing edit actions due to flask-admin upgrade
- Fix typo in comment in prioritize_queued method
- Add HipchatOperator
- Include all example dags in backfill unit test
- Make sure skipped jobs are actually skipped
- Fixing a broken example dag, example_skip_dag.py
- Add consistent and thorough signal handling and logging
- Allow Operators to specify SKIPPED status internally
- Update docstring for executor trap unit test
- Doc: explain the usage of Jinja templating for templated params
- Don't schedule runs before the DAG's start_date
- Fix infinite retries with pools, with test
- Fix handling of deadlocked jobs
- Show only Airflow's deprecation warnings
- Set DAG_FOLDER for unit tests
- Missing comma in setup.py
- Deprecate *args and **kwargs in BaseOperator
- Raise deep scheduler exceptions to force a process restart.
- Change inconsistent example DAG owners
- Fix module path of send_email_smtp in configuration
- added Gentner Lab to list of users
- Increase timeout time for unit test
- Fix reading strings from conf
- CHORE - Remove Trailing Spaces
- Fix SSHExecuteOperator crash when using a custom ssh port
- Add note about airflow components to template
- Rewrite BackfillJob logic for clarity
- Add unit tests
- Fix miscellaneous bugs and clean up code
- Fix logic for determining DagRun states
- Make SchedulerJob not run EVERY queued task
- Improve BackfillJob handling of queued/deadlocked tasks
- Introduce ignore_depends_on_past parameters
- Use Popen with CeleryExecutor
- Rename user table to users to avoid conflict with postgres
- Beware of negative pool slots.
- Add support for calling_format from boto to S3_Hook
- Add pypi meta data and sync version number
- Set dags_are_paused_at_creation's default value to True
- Resurface S3Log class eaten by rebase/push -f
- Add missing session.commit() at end of initdb
- Validate that subdag tasks have pool slots available, and test
- Use urlparse for remote GCS logs, and add unit tests
- Make webserver worker timeout configurable
- Fixed scheduling for @once interval
- Use psycopg2's API for serializing postgres cell values
- Make the provide_session decorator more robust
- update link to Lyft's website
- use num_shards instead of partitions to be consistent with batch ingestion
- Add documentation links to README
- Update docs with separate configuration section
- Fix airflow.utils deprecation warning code being Python 3 incompatible
- Extract dbapi cell serialization into its own method
- Set Postgres autocommit as supported only if server version is < 7.4
- Use refactored utils module in unit test imports
- Add changelog for 1.7.0
- Use LocalExecutor on Travis if possible
- remove unused logging,errno, MiniHiveCluster imports
- remove extra import of logging lib
- Fix required gcloud version
- Refactoring utils into smaller submodules
- Properly measure number of task retry attempts
- Add function to get configuration as dict, plus unit tests
- Merge branch 'master' into hivemeta_sasl
- Add wiki link to README.md
- [hotfix] make email.Utils > email.utils for py3
- Add the missing "Date" header to the warning e-mails
- Add the missing "Date" header to the warning e-mails
- Check name of SubDag class instead of class itself
- [hotfix] removing repo_token from .coveralls.yml
- Set the service_name in coverals.yml
- Fixes #1223
- Update Airflow docs for remote logging
- Add unit tests for trapping Executor errors
- Make sure Executors properly trap errors
- Fix HttpOpSensorTest to use fake resquest session
- Linting
- Add an example on pool usage in the documentation
- Add two methods to bigquery hook's base cursor: run_table_upsert, which adds a table or updates an existing table; and run_grant_dataset_view_access, which grants view access to a given dataset for a given table.
- Tasks references upstream and downstream tasks using strings instead of references
- Fix typos in models.py
- Fix broken links in documentation
- [hotfix] fixing the Scheduler CLI to make dag_id optional
- Update link to Common Pitfalls wiki page in README
- Allow disabling periodic committing when inserting rows with DbApiHook
- added Glassdoor to "who uses airflow"
- Fix typo preventing from launching webserver
- Documentation badge
- Fixing ISSUE_TEMPLATE name to include .md suffix
- Adding an ISSUE_TEMPLATE to ensure that issues are adequately defined
- Linting & debugging
- Refactoring the CLI to be data-driven
- Updating the Bug Reporting protocol in the Contributing.md file
- Fixing the docs
- clean up references to old session
- remove session reference
- resolve conflict
- clear xcom data when task instance starts
- replace main_session with @provide_session
- Add extras to installation.rst
- Changes to Contributing to reflect more closely the current state of development.
- Modifying README to link to the wiki committer list
- docs: fixes a spelling mistake in default config
- Set killMode to 'control-group' for webservice.service
- Set KillMode to 'control-group' for worker.service
- Linting
- Fix WebHdfsSensor
- Adding more licenses to pass checks
- fixing landscape's config
- [hotfix] typo that made it in master
- [hotfix] fixing landscape requirement detection
- Make testing on hive conditional
- Merge remote-tracking branch 'upstream/master' into minicluster
- Update README.md
- Throwing in a few license to pass the build
- Adding a reqs.txt for landscape.io
- Pointing to a reqs file
- Some linting
- Adding a .landscape.yml file
- badge for pypi version
- Add license and ignore for sql and csv
- Use correct connection id
- Use correct table name
- Provide data for ci tests
- new badge for showing staleness of reqs
- removing requirements.txt as it is uni-dimensional
- Make it work on py3
- Remove decode for logging
- Also keep py2 compatible
- More py3 fixes
- Convert to bytes for py3 compat
- Make sure to be py3 compatible
- Use unicodecsv to make it py3 compatible
- Replace tab with spaces Remove unused import
- Merge remote-tracking branch 'upstream/master'
- Support decimal types in MySQL to GCS
- Make sure to write binary as string can be unicode
- Ignore metastore
- More impyla fixes
- Test HivemetaStore if python 2
- Allow users to set hdfs_namenode_principal in HDFSHook config
- Add tests for Hiveserver2 and fix some issues from impyla
- Merge branch 'impyla' into minicluster
- This patch allows for testing of hive operators and hooks. Sasl is used (NoSasl in connection string is not possible). Tests have been adjusted.
- Treat SKIPPED and SUCCESS the same way when evaluating depends_on_past=True
- fix bigquery hook
- version cap for gcp_api
- Fix typo when returning VerticaHook
- Adding fernet key to use it as part of stdout commands
- Adding support for ssl parameters. (picking up from jthomas123)
- more detail in error message.
- make sure paths don't conflict bc of trailing /
- change gcs_hook to self.hook
- refactor remote log read/write and add GCS support
- Only use multipart upload in S3Hook if file is large enough
- Merge branch 'airbnb/master'
- Add GSSAPI SASL to HiveMetaStoreHook.
- Add warning for deprecated setting
- Use kerberos_service_name = 'hive' as standard instead of 'impala'.
- Use GSSAPI instead of KERBEROS and provide backwards compatibility
- ISSUE-1123 Use impyla instead of pyhs2
- set celery_executor to use queue name as exchange