Skip to content

Commit

Permalink
Merge pull request #83 from developmentseed/develop
Browse files Browse the repository at this point in the history
version 0.7.0
  • Loading branch information
Scisco committed May 29, 2015
2 parents 87784f3 + 507edb2 commit 5dfbaf8
Show file tree
Hide file tree
Showing 27 changed files with 1,406 additions and 859 deletions.
53 changes: 28 additions & 25 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,30 +1,33 @@
language: python

python:
- '2.7'

# Setup anaconda
- '2.7'
before_install:
- sudo add-apt-repository -y ppa:ubuntugis/ppa
- sudo apt-get update -qq
- sudo apt-get install libgdal1h gdal-bin libgdal-dev libcurl4-gnutls-dev
- wget http://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh -O miniconda.sh
- chmod +x miniconda.sh
- ./miniconda.sh -b
- export PATH=/home/travis/miniconda/bin:$PATH
- conda update --yes conda
# The next couple lines fix a crash with multiprocessing on Travis and are not specific to using Miniconda
- sudo rm -rf /dev/shm
- sudo ln -s /run/shm /dev/shm

# Install packages
- sudo add-apt-repository -y ppa:ubuntugis/ppa
- sudo apt-get update -qq
- sudo apt-get install libgdal1h gdal-bin libgdal-dev libcurl4-gnutls-dev
- wget http://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh -O miniconda.sh
- chmod +x miniconda.sh
- "./miniconda.sh -b"
- export PATH=/home/travis/miniconda/bin:$PATH
- conda update --yes conda
- sudo rm -rf /dev/shm
- sudo ln -s /run/shm /dev/shm
install:
- conda install --yes python=$TRAVIS_PYTHON_VERSION numpy scipy matplotlib scikit-image six nose dateutil
- conda install --yes -c https://conda.binstar.org/osgeo gdal
- pip install --install-option="--no-cython-compile" cython
- pip uninstall requests --yes
- pip install requests==2.5.3
- pip install -r requirements/travis.txt

- conda install --yes python=$TRAVIS_PYTHON_VERSION numpy scipy matplotlib scikit-image
six nose dateutil
- conda install --yes -c https://conda.binstar.org/osgeo gdal
- pip install --install-option="--no-cython-compile" cython
- pip uninstall requests --yes
- pip install requests==2.5.3
- pip install -r requirements/travis.txt
script:
- nosetests
- nosetests
deploy:
provider: pypi
user: devseed
password:
secure: WtawFW/999XYszmZfj1Qk82l00OSyP2JBVOOGCERrW1gVO7MYtYsgP31HKRSzNTCTHJNVDpdK4WZWY6zPQqC3l2UfWYYsvRn0hCoI8AJxE5VCUEg6Ccpe6fMJuhp1pq6Zy7yrfBSZcOB9aqSHLBJsunD2o3mNlTC8WV8vNK74ck=
on:
tags: true
repo: developmentseed/landsat-util
branch: master
12 changes: 12 additions & 0 deletions CHANGES.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,18 @@
Changes
=======

0.7.0 (2015-05-29)
------------------
- New documentation
- Deployed to readthedocs
- Automate deployment to pypi
- Adds docker support
- skip unzipping if images already unzipped
- add force-unzip flag
- fix a bug where multiple downloads was not followed by multiple process #81
- fix a bug where if scenes was downloaded from google instead of aws, process failed #84
- download band 8 when pansharpen fixes #73

0.6.3 (2015-04-29)
------------------
- adjust lower rescaling bound, closes #66 for now
Expand Down
4 changes: 4 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
FROM ubuntu:14.10
RUN apt-get -y update
RUN apt-get install --yes python-pip python-numpy python-scipy libgdal-dev libatlas-base-dev gfortran libfreetype6-dev libglib2.0-dev zlib1g-dev python-pycurl
RUN pip install landsat-util
177 changes: 17 additions & 160 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,183 +4,40 @@ Landsat-util
.. image:: https://travis-ci.org/developmentseed/landsat-util.svg?branch=v0.5
:target: https://travis-ci.org/developmentseed/landsat-util

.. image:: https://pypip.in/version/landsat-util/badge.svg
:target: https://pypi.python.org/pypi/landsat-util/
:alt: Latest Version
.. image:: https://badge.fury.io/py/landsat-util.svg
:target: http://badge.fury.io/py/landsat-util

.. image:: https://pypip.in/download/landsat-util/badge.svg
.. image:: https://img.shields.io/pypi/dm/landsat-util.svg
:target: https://pypi.python.org/pypi/landsat-util/
:alt: Downloads

.. image:: https://pypip.in/wheel/landsat-util/badge.svg
:target: https://pypi.python.org/pypi/landsat-util/
:alt: Wheel Status

.. image:: https://pypip.in/license/landsat-util/badge.svg
.. image:: https://img.shields.io/pypi/l/landsat-util.svg
:target: https://pypi.python.org/pypi/landsat-util/
:alt: License

Landsat-util is a command line utility that makes it easy to search, download, and process Landsat imagery.

This tool uses Development Seed's `API for Landsat Metadata <https://github.com/developmentseed/landsat-api>`_.

This API is accessible here: https://api.developmentseed.org/landsat

You can also run your own API and connect it to this tool.

Installation
============

**On Mac**

``$: pip install landsat-util``

**On Ubuntu 14.04**

Use pip to install landsat-util. If you are not using virtualenv, you might have to run ``pip`` as ``sudo``.

``$: sudo apt-get update``

``$: sudo apt-get install python-pip python-numpy python-scipy libgdal-dev libatlas-base-dev gfortran``

``$: pip install landsat-util``

**On Other systems**

Make sure Python setuptools is installed.

``$: python setup.py numpy six``

``$: python setup.py install``


**To Upgrade**

``$: pip install -U landsat-util``

If you have installed previous version of landsat using brew, first run:

``$: brew uninstall landsat-util``

**To Test**

``$: pip install -U requirements/dev.txt``

``$: nosetests``

Or

``$: python setup.py test``

Overview: What can landsat-util do?
====================================

Landsat-util has three main functions:

- **Search** for landsat tiles based on several search parameters.
- **Download** landsat images.
- **Image processing** and pan sharpening on landsat images.

These three functions have to be performed separately.

**Help**: Type ``landsat -h`` for detailed usage parameters.

Step 1: Search
===============

Search returns information about all landsat tiles that match your criteria. This includes a link to an unprocessed preview of the tile. The most important result is the tile's *sceneID*, which you will need to download the tile (see step 2 below).

Search for landsat tiles in a given geographical region, using any of the following:

- **Paths and rows**: If you know the paths and rows you want to search for.
- **Latidue and Longitude**: If you need the latitude and longitude of the point you want to search for.

Additionally filter your search using the following parameters:

- **Start and end dates** for when imagery was taken
- **Maximum percent cloud cover** (default is 20%)

**Examples of search**:

Search by path and row:

``$: landsat search --cloud 4 --start "january 1 2014" --end "january 10 2014" -p 009,045``

Search by latitude and longitude:

``$: landsat search --lat 38.9004204 --lon -77.0237117``


Step 2: Download
=================

You can download tiles using their unique sceneID, which you get from landsat search.

Landsat-util will download a zip file that includes all the bands. You have the option of specifying the bands you want to down. In this case, landsat-util only downloads those bands if they are available online.

**Examples of download**:

Download images by their custom sceneID, which you get from landsat search:

``$: landsat download LC80090452014008LGN00``

Download only band 4, 3 and 2 for a particular sceneID:

``$: landsat download LC80090452014008LGN00 --bands 432``

Download multiple sceneIDs:

``$: landsat download LC80090452014008LGN00 LC80090452015008LGN00 LC80090452013008LGN00``

Step 3: Image processing
=========================

You can process your downloaded tiles with our custom image processing algorithms. In addition, you can choose to pansharpen your images and specify which bands to process.

**Examples of image processing**:

Process images that are already downloaded. Remember, the program accepts both zip files and unzipped folders:

``$: landsat process path/to/LC80090452014008LGN00.tar.bz``

If unzipped:

``$: landsat process path/to/LC80090452014008LGN00``

Specify bands 3, 5 and 1:

``$: landsat process path/to/LC80090452014008LGN00 --bands 351``

Process *and* pansharpen a downloaded image:

``$: landsat process path/to/LC80090452014008LGN00.tar.bz --pansharpen``


Important Notes
===============
Landsat-util is a command line utility that makes it easy to search, download, and process Landsat imagery.

- All downloaded and processed images are stored at your home directory in landsat forlder: ``~/landsat``
Docs
+++++

- The image thumbnail web address that is included in the results can be used to make sure that clouds are not obscuring the subject of interest. Run the search again if you need to narrow down your result and then start downloading images. Each image is usually more than 700mb and it might takes a very long time if there are too many images to download
For full documentation visit: http://landsat-util.readthedocs.org/

- Image processing is a very heavy and resource consuming task. Each process takes about 5-10 mins. We recommend that you run the processes in smaller badges. Pansharpening, while increasing image resolution 2x, substantially increases processing time.
To run the documentation locally::

- Landsat-util requires at least 2GB of Memory (RAM).
$ cd docs
$ make html

Recently Added
+++++++++++++++
Recently Added Features
+++++++++++++++++++++++

- Add longitude latitude search
- Improve console output
- Add more color options such as false color, true color, etc.

Change Log
+++++++++

See `CHANGES.txt <CHANGES.rst>`_.

To Do List
++++++++++

- Add Sphinx Documentation
- Add capacity for NDVI output
- Add alternative projections (currently only option is default web-mercator; EPSG: 3857)
- Connect search to Google Address API
- Include 16-bit image variant in output
- Add support for color correct looping over multiple compressed inputs (currently just 1)
37 changes: 0 additions & 37 deletions Vagrantfile

This file was deleted.

25 changes: 20 additions & 5 deletions docs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .

.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest coverage gettext

help:
@echo "Please use \`make <target>' where <target> is one of"
Expand All @@ -30,6 +30,7 @@ help:
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " applehelp to make an Apple Help Book"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
Expand All @@ -45,6 +46,7 @@ help:
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
@echo " coverage to run coverage check of the documentation (if enabled)"

clean:
rm -rf $(BUILDDIR)/*
Expand Down Expand Up @@ -85,17 +87,25 @@ qthelp:
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/landsat.qhcp"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/Landsat-util.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/landsat.qhc"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/Landsat-util.qhc"

applehelp:
$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
@echo
@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
@echo "N.B. You won't be able to view it unless you put it in" \
"~/Library/Documentation/Help or install it in your application" \
"bundle."

devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/landsat"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/landsat"
@echo "# mkdir -p $$HOME/.local/share/devhelp/Landsat-util"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/Landsat-util"
@echo "# devhelp"

epub:
Expand Down Expand Up @@ -166,6 +176,11 @@ doctest:
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."

coverage:
$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
@echo "Testing of coverage in the sources finished, look at the " \
"results in $(BUILDDIR)/coverage/python.txt."

xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
Expand Down
Loading

0 comments on commit 5dfbaf8

Please sign in to comment.