Merge pull request #93 from GeospatialPython/karimbahgat-patch-dtypesdocs

Docs improvements and dtype fixes

Author: karimbahgat

README.md

@@ -1,6 +1,24 @@
 ## Contents
 
-[TOC]
+[Overview](#overview)
+
+[Examples](#examples)
+- [Reading Shapefiles](#reading-shapefiles)
+  - [Reading Shapefiles from File-Like Objects](#reading-shapefiles-from-file-like-objects)
+  - [Reading Geometry](#reading-geometry)
+  - [Reading Records](#reading-records)
+  - [Reading Geometry and Records Simultaneously](#reading-geometry-and-records-simultaneously)
+- [Writing Shapefiles](#writing-shapefiles)
+  - [Setting the Shape Type](#setting-the-shape-type)
+  - [Geometry and Record Balancing](#geometry-and-record-balancing)
+  - [Adding Geometry](#adding-geometry)
+  - [Creating Attributes](#creating-attributes)
+  - [File Names](#file-names)
+  - [Saving to File-Like Objects](#saving-to-file-like-objects)
+- [Editing Shapefiles](#editing-shapefiles)
+  - [Geometry and Record Balancing](#geometry-and-record-balancing)
+- [Python \_\_geo_interface\_\_](#python-\_\_geo\_interface\_\_)
+- [Testing](#testing)
 
 # Overview
 
@@ -385,17 +403,17 @@ data lines up with the geometry data. For example:
 ### Adding Geometry
 
 Geometry is added using one of three methods: "null", "point", or "poly". The
-"null" method is used for null shapes, "point" is used for point shapes, and
-"poly" is used for everything else.
+"null" method is used for null shapes, "point" is used for point shapes, "line" for lines, and
+"poly" is used for polygons and everything else.
 
 **Adding a Point shape**
 
 Point shapes are added using the "point" method. A point is specified by an x,
 y, and optional z (elevation) and m (measure) value.
 
 
-    >>> w = shapefile.Writer()
-
+    >>> w = shapefile.Writer(shapefile.POINTM)
+	
     >>> w.point(122, 37) # No elevation or measure values
 
     >>> w.shapes()[0].points
@@ -405,21 +423,45 @@ y, and optional z (elevation) and m (measure) value.
 
     >>> w.shapes()[1].points
     [[118, 36, 4, 8]]
+	
+	>>> w.field('FIRST_FLD', 'C')
+	>>> w.field('SECOND_FLD', 'C')
+	
+	>>> w.save('shapefiles/test/point')
 
-**Adding a Poly shape**
+**Adding a Polygon shape**
 
-"Poly" shapes can be either polygons or lines. Shapefile polygons must have at
+Shapefile polygons must have at
 least 4 points and the last point must be the same as the first. PyShp
-automatically enforces closed polygons. A line must have at least two points.
-Because of the similarities between these two shape types they are created
-using a single method called "poly".
+automatically enforces closed polygons. 
 
 
     >>> w = shapefile.Writer()
 
-    >>> w.poly(shapeType=3, parts=[[[122,37,4,9], [117,36,3,4]], [[115,32,8,8],
+    >>> w.poly(parts=[[[122,37,4,9], [117,36,3,4]], [[115,32,8,8],
     ... [118,20,6,4], [113,24]]])
 
+	>>> w.field('FIRST_FLD', 'C')
+	>>> w.field('SECOND_FLD', 'C')
+	
+	>>> w.save('shapefiles/test/polygon')
+
+**Adding a Line shape**
+
+A line must have at least two points.
+Because of the similarities between polygon and line types it is possible to create
+a line shape using either the "line" or "poly" method.
+	
+	>>> w = shapefile.Writer()
+	
+    >>> w.line(parts=[[[1,5],[5,5],[5,1],[3,3],[1,1]]])
+    >>> w.poly(parts=[[[1,3],[5,3]]], shapeType=shapefile.POLYLINE)
+	
+	>>> w.field('FIRST_FLD', 'C')
+	>>> w.field('SECOND_FLD', 'C')
+	
+	>>> w.save('shapefiles/test/polygon')
+	
 **Adding a Null shape**
 
 Because Null shape types (shape type 0) have no geometry the "null" method is
@@ -439,51 +481,114 @@ The writer object's shapes list will now have one null shape:
 
 Creating attributes involves two steps. Step 1 is to create fields to contain
 attribute values and step 2 is to populate the fields with values for each
-shape record.
-
-The following attempts to create a complete shapefile.  The attribute and
-field names are not very creative:
-
-
-    >>> w = shapefile.Writer(shapefile.POINT)
-    >>> w.point(1,1)
-    >>> w.point(3,1)
-    >>> w.point(4,3)
-    >>> w.point(2,2)
-    >>> w.field('FIRST_FLD')
-    >>> w.field('SECOND_FLD','C','40')
-    >>> w.record('First','Point')
-    >>> w.record('Second','Point')
-    >>> w.record('Third','Point')
-    >>> w.record('Fourth','Point')
-    >>> w.save('shapefiles/test/point')
-
-    >>> w = shapefile.Writer(shapefile.POLYGON)
-    >>> w.poly(parts=[[[1,5],[5,5],[5,1],[3,3],[1,1]]])
-    >>> w.field('FIRST_FLD','C','40')
-    >>> w.field('SECOND_FLD','C','40')
-    >>> w.record('First','Polygon')
-    >>> w.save('shapefiles/test/polygon')
-
-    >>> w = shapefile.Writer(shapefile.POLYLINE)
-    >>> w.line(parts=[[[1,5],[5,5],[5,1],[3,3],[1,1]]])
-    >>> w.poly(parts=[[[1,3],[5,3]]], shapeType=shapefile.POLYLINE)
-    >>> w.field('FIRST_FLD','C','40')
-    >>> w.field('SECOND_FLD','C','40')
-    >>> w.record('First','Line')
-    >>> w.record('Second','Line')
-    >>> w.save('shapefiles/test/line')
+shape record. 
 
-You can also add attributes using keyword arguments where the keys are field
-names.
+There are several different field types, all of which support storing None values as NULL. 
 
+Text fields are created using the 'C' type, and the third 'size' argument can be customized to the expected
+length of text values to save space:
 
-    >>> w = shapefile.Writer(shapefile.POLYLINE)
-    >>> w.line(parts=[[[1,5],[5,5],[5,1],[3,3],[1,1]]])
-    >>> w.field('FIRST_FLD','C','40')
-    >>> w.field('SECOND_FLD','C','40')
-    >>> w.record(FIRST_FLD='First', SECOND_FLD='Line')
-    >>> w.save('shapefiles/test/line')
+
+    >>> w = shapefile.Writer()
+    >>> w.field('TEXT', 'C')
+	>>> w.field('SHORT_TEXT', 'C', size=5)
+	>>> w.field('LONG_TEXT', 'C', size=250)
+	>>> w.null()
+    >>> w.record('Hello', 'World', 'World'*50)
+    >>> w.save('shapefiles/test/dtype')
+	
+	>>> r = shapefile.Reader('shapefiles/test/dtype')
+	>>> assert r.record(0) == ['Hello', 'World', 'World'*50]
+
+Date fields are created using the 'D' type, and can be created using either 
+date objects, lists, or a YYYYMMDD formatted string. 
+Field length or decimal have no impact on this type:
+
+
+	>>> from datetime import date
+    >>> w = shapefile.Writer()
+    >>> w.field('DATE', 'D')
+	>>> w.null()
+	>>> w.null()
+	>>> w.null()
+	>>> w.null()
+    >>> w.record(date(1998,1,30))
+	>>> w.record([1998,1,30])
+	>>> w.record('19980130')
+	>>> w.record(None)
+    >>> w.save('shapefiles/test/dtype')
+	
+	>>> r = shapefile.Reader('shapefiles/test/dtype')
+	>>> assert r.record(0) == [date(1998,1,30)]
+	>>> assert r.record(1) == [date(1998,1,30)]
+	>>> assert r.record(2) == [date(1998,1,30)]
+	>>> assert r.record(3) == [None]
+
+Numeric fields are created using the 'N' type (or the 'F' type, which is exactly the same). 
+By default the fourth decimal argument is set to zero, essentially creating an integer field. 
+To store floats you must set the decimal argument to the precision of your choice. 
+To store very large numbers you must increase the field length size to the total number of digits 
+(including comma and minus). 
+
+
+    >>> w = shapefile.Writer()
+	>>> w.field('INT', 'N')
+    >>> w.field('LOWPREC', 'N', decimal=2)
+	>>> w.field('MEDPREC', 'N', decimal=10)
+	>>> w.field('HIGHPREC', 'N', decimal=30)
+	>>> w.field('FTYPE', 'F', decimal=10)
+	>>> w.field('LARGENR', 'N', 101)
+	>>> nr = 1.3217328
+	>>> w.null()
+	>>> w.null()
+    >>> w.record(INT=int(nr), LOWPREC=nr, MEDPREC=nr, HIGHPREC=-3.2302e-25, FTYPE=nr, LARGENR=int(nr)*10**100)
+	>>> w.record(None, None, None, None, None, None)
+    >>> w.save('shapefiles/test/dtype')
+	
+	>>> r = shapefile.Reader('shapefiles/test/dtype')
+	>>> r.record(0)
+	[1, 1.32, 1.3217328, -3.2302e-25, 1.3217328, 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000L]
+	>>> r.record(1)
+	[None, None, None, None, None, None]
+
+	
+Finally, we can create boolean fields by setting the type to 'L'. 
+This field can take True or False values, or any string whose first character is one of YyTt (True) or NnFf (False). 
+None is interpreted as missing. 
+
+
+    >>> w = shapefile.Writer()
+	>>> w.field('BOOLEAN', 'L')
+	>>> w.null()
+	>>> w.null()
+	>>> w.null()
+	>>> w.null()
+	>>> w.null()
+    >>> w.record(True)
+	>>> w.record("Yes")
+	>>> w.record(False)
+	>>> w.record("No")
+	>>> w.record(None)
+    >>> w.save('shapefiles/test/dtype')
+	
+	>>> r = shapefile.Reader('shapefiles/test/dtype')
+	>>> assert r.record(0) == [True]
+	>>> assert r.record(1) == [True]
+	>>> assert r.record(2) == [False]
+	>>> assert r.record(3) == [False]
+	>>> assert r.record(4) == [None]
+	
+	
+You can also add attributes using keyword arguments where the keys are field names.
+
+	>>> w = shapefile.Writer()
+	>>> w.field('FIRST_FLD','C','40')
+	>>> w.field('SECOND_FLD','C','40')
+	>>> w.record('First', 'Line')
+	>>> w.record(FIRST_FLD='First', SECOND_FLD='Line')
+	>>> assert w.records[0] == w.records[1]
+	
+	
 
 ### File Names
 

shapefile.py

@@ -926,15 +926,17 @@ def __dbfRecords(self):
         for record in self.records:
             if not self.fields[0][0].startswith("Deletion"):
                 f.write(b(' ')) # deletion flag
-            for (fieldName, fieldType, size, dec), value in zip(self.fields, record):
+            for (fieldName, fieldType, size, deci), value in zip(self.fields, record):
                 fieldType = fieldType.upper()
                 size = int(size)
                 if fieldType in ("N","F"):
                     # numeric or float: number stored as a string, right justified, and padded with blanks to the width of the field.
                     if value in MISSING:
                         value = str("*"*size) # QGIS NULL
+                    elif not deci:
+                        value = format(value, "d")[:size].rjust(size) # caps the size if exceeds the field size
                     else:
-                        value = str(value).rjust(size)
+                        value = format(value, ".%sf"%deci)[:size].rjust(size) # caps the size if exceeds the field size
                 elif fieldType == "D":
                     # date: 8 bytes - date stored as a string in the format YYYYMMDD.
                     if isinstance(value, date):
@@ -946,7 +948,7 @@ def __dbfRecords(self):
                     elif isinstance(value, str) and len(value) == 8:
                         pass # value is already a date string
                     else:
-                        raise ShapefileException("Date values must be either a datetime.date object, a list, or a missing value of None or ''.")
+                        raise ShapefileException("Date values must be either a datetime.date object, a list, a YYYYMMDD string, or a missing value.")
                 elif fieldType == 'L':
                     # logical: 1 byte - initialized to 0x20 (space) otherwise T or F.
                     if value in MISSING:
@@ -967,9 +969,9 @@ def null(self):
         """Creates a null shape."""
         self._shapes.append(_Shape(NULL))
 
-    def point(self, x, y, z=0, m=0):
+    def point(self, x, y, z=0, m=0, shapeType=POINT):
         """Creates a point shape."""
-        pointShape = _Shape(self.shapeType)
+        pointShape = _Shape(shapeType)
         pointShape.points.append([x, y, z, m])
         self._shapes.append(pointShape)
 
@@ -1060,7 +1062,7 @@ def saveShp(self, target):
             target = os.path.splitext(target)[0] + '.shp'
         if self.shapeType is None:
             # autoset file type to first non-null geometry
-            self.shapeType = next((s.shapeType for s in self._shapes if s.shapeType != NULL), NULL)
+            self.shapeType = next((s.shapeType for s in self._shapes if s.shapeType), NULL)
         self.shp = self.__getFileObj(target)
         self.__shapefileHeader(self.shp, headerType='shp')
         self.__shpRecords()