Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for camera back "Hasselblad CFV-50c" (100% equivalent to the H5D-50c) #622

Merged
merged 1 commit into from
Jan 25, 2024
Merged

Conversation

mristuccia
Copy link
Contributor

Added camera back "Hasselblad CFV-50c" which is completely equivalent to the "Hasselblad H5D-50c" (also tested by changing camera model on exif file from H5D-50c to CFV-50c).

…o the H5D-50c)

Added camera back "Hasselblad CFV-50c" which is completely equivalent to the "Hasselblad H5D-50c" (also tested by changing camera model on exif file from H5D-50c to CFV-50c).
@mristuccia mristuccia requested a review from LebedevRI as a code owner January 25, 2024 22:04
@LebedevRI
Copy link
Member

Is this the same camera as Hasselblad CFV-50c/200, for which the two samples were just uploaded?
If so, the names don't match: RawSpeed:Unable to find camera in database: 'Hasselblad' 'Hasselblad CFV-50c/200' ''

@mristuccia
Copy link
Contributor Author

mristuccia commented Jan 25, 2024

Is this the same camera as Hasselblad CFV-50c/200, for which the two samples were just uploaded? If so, the names don't match: RawSpeed:Unable to find camera in database: 'Hasselblad' 'Hasselblad CFV-50c/200' ''

I don't think so. EXIF camera model on CFV-50c FFF raw files says "Hasselblad CFV-50c" and not "Hasselblad CFV-50c/200". Such raw files don't load on Darktable. If I change the EXIF camera model with an exif editor to "Hasselblad H5D-50c" the raw files is loaded correctly on Darktable.

Honestly, I don't think a camera model named "Hasselblad CFV-50c/200" even exist(ed) on Hasselblad portfolio.

I did not load any example up until now. If needed, where do I have to upload sample RAW files?

@LebedevRI
Copy link
Member

I see. Then please contribute the full sample set to https://raw.pixls.us/
As far as i can tell from camera's PDF manual, there is only a single RAW format,
so a single sample is enough.
Please, make it something like a bright daylight landscape, not dim indoors shot.

@LebedevRI
Copy link
Member

Also, please be sure to upload the file manually copied straight from the card,
without any modifications whatsoever by anything.

@mristuccia thank you!

@mristuccia
Copy link
Contributor Author

Also, please be sure to upload the file manually copied straight from the card, without any modifications whatsoever by anything.

@mristuccia thank you!

Sure! Just uploaded. Hope this is what you need.
Thank you for the support!

@LebedevRI LebedevRI merged commit 1d13d3d into darktable-org:develop Jan 25, 2024
1 check passed
@LebedevRI
Copy link
Member

@mristuccia thank you for contributing the sample!

@mristuccia thank you!

@mristuccia
Copy link
Contributor Author

@LebedevRI thank you for the support.
Hope I can use Darktable with my CFV-50c raw files soon. 😊

@mristuccia mristuccia deleted the mristuccia-patch-1 branch January 25, 2024 23:07
@LebedevRI
Copy link
Member

LebedevRI commented Jan 25, 2024

Well, if you can build dt yourself, you just need to update the src/external/rawspeed to use develop branch,
and locally apply darktable-org/darktable#16185 so it builds.

@kmilos
Copy link
Collaborator

kmilos commented Jan 25, 2024

I'd much rather we went w/ aliases for Hasselblads like we did in #621 etc. Only not clear if this should be an alias for H5D-50c or X1D...

Also note that the model string usually changes depending whether it is the .3FR raw from the card directly or .FFF raw via Phocus software tethered transfer.

This PR and the uploaded sample only cover the .FFF option.

@mristuccia
Copy link
Contributor Author

mristuccia commented Jan 25, 2024

I'd much rather we went w/ aliases for Hasselblads like we did in #621 etc. Only not clear if this should be an alias for H5D-50c or X1D...

Definitely not the X1D. X1D is more the equivalent of the CFV-50c Mk II (or vice versa).
H5D-50c and CFV-50c came out from the same project. One is a general purpose digital back which can be attached to the old 500 series cameras. The H5D-50c is the official digital back equivalent for the newest autofocus H series models. Internally they should be the same AFAIK, but I would not alias one into the other as they are two different products in the Hasselblad Catalog.

Also note that the model string usually changes depending whether it is the .3FR raw from the card directly or .FFF raw via Phocus software tethered transfer.

This PR and the uploaded sample only cover the .FFF option.

3FR files cannot be edited in the standard Hasselblad Phocus RAW developer, they need to be imported to be edited. The import procedure turns them into FFFs, which soon after the import/tether are still considered untouched originals.
So, by Hasselblad design 3FRs should not be used for RAW editing, but if you think this design rule should be ignored and 3FRs should be covered as well, please explain to me what I need to do and I'll try to submit a new PR for that.

@LebedevRI
Copy link
Member

I'd much rather we went w/ aliases for Hasselblads like we did in #621 etc. Only not clear if this should be an alias for H5D-50c or X1D...

I mean, it's not like things are set in stone afterwards.

...

I guess, you need to take the https://raw.pixls.us/getfile.php/7199/nice/Hasselblad%20-%20Hasselblad%20CFV-50c%20-%2016bit%20(4:3).fff, import into whatever software, and then upload the resulting "imported" "raw" again to RPU,
for a start.

@kmilos
Copy link
Collaborator

kmilos commented Jan 26, 2024

Definitely not the X1D. X1D is more the equivalent of the CFV-50c Mk II (or vice versa).

CVF II 50C is based on X1D II 50C AFAICT, not the original X1D. But all three seem to have the same sensor.

H5D-50c and CFV-50c came out from the same project. One is a general purpose digital back which can be attached to the old 500 series cameras. The H5D-50c is the official digital back equivalent for the newest autofocus H series models.

Thanks for the background!

Internally they should be the same AFAIK, but I would not alias one into the other as they are two different products in the Hasselblad Catalog.

Oh, you'd still see them as separate models in darktable, it's just that internal technical debt in maintaining rawspeed database, dt noise profiles etc. is reduced.

3FR files cannot be edited in the standard Hasselblad Phocus RAW developer, they need to be imported to be edited. The import procedure turns them into FFFs, which soon after the import/tether are still considered untouched originals.
So, by Hasselblad design 3FRs should not be used for RAW editing, but if you think this design rule should be ignored and 3FRs should be covered as well, please explain to me what I need to do and I'll try to submit a new PR for that.

AFAICT this is more of an recommendation than a rule. Adobe Camera Raw certainly lists 3FR as supported for example, and we seem to have no problem loading either 3FR or FFF for the supported models in rawspeed and dt.

For starters, it would be good to also upload the 3FR that was the source for the the FFF you have already uploaded (or upload a new matching pair).

@LebedevRI I think the other Hasselblad - Hasselblad CFV-50c/200 - 16bit (4:3).3FR sample we have came from attaching the back to a 200 series body as opposed to the 500 one.

@kmilos
Copy link
Collaborator

kmilos commented Jan 26, 2024

So, if your 3FR comes out as CFV-50c/500 (I am indeed getting Exif hits on a web search), these are the changes I propose:

diff --git a/data/cameras.xml b/data/cameras.xml
index 569bd69d..b557bb7d 100644
--- a/data/cameras.xml
+++ b/data/cameras.xml
@@ -17286,6 +17286,11 @@
 		</CFA>
 		<Crop x="56" y="104" width="-56" height="0"/>
 		<Sensor black="256" white="62914"/>
+		<Aliases>
+			<Alias id="CFV-50c">Hasselblad CFV-50c</Alias>
+			<Alias id="CFV-50c">Hasselblad CFV-50c/200</Alias>
+			<Alias id="CFV-50c">Hasselblad CFV-50c/500</Alias>
+		</Aliases>
 		<ColorMatrices>
 			<ColorMatrix planes="3">
 				<ColorMatrixRow plane="0">4932 -835 141</ColorMatrixRow>
@@ -17294,24 +17299,6 @@
 			</ColorMatrix>
 		</ColorMatrices>
 	</Camera>
-	<Camera make="Hasselblad" model="Hasselblad CFV-50c">
-		<ID make="Hasselblad" model="CFV-50c">Hasselblad 50-15-Coated5</ID>
-		<CFA width="2" height="2">
-			<Color x="0" y="0">RED</Color>
-			<Color x="1" y="0">GREEN</Color>
-			<Color x="0" y="1">GREEN</Color>
-			<Color x="1" y="1">BLUE</Color>
-		</CFA>
-		<Crop x="56" y="104" width="-56" height="0"/>
-		<Sensor black="256" white="62914"/>
-		<ColorMatrices>
-			<ColorMatrix planes="3">
-				<ColorMatrixRow plane="0">4932 -835 141</ColorMatrixRow>
-				<ColorMatrixRow plane="1">-4878 11868 3437</ColorMatrixRow>
-				<ColorMatrixRow plane="2">-1138 1961 7067</ColorMatrixRow>
-			</ColorMatrix>
-		</ColorMatrices>
-	</Camera>	
 	<Camera make="Hasselblad" model="Flash Sync">
 		<ID make="Hasselblad" model="CF132">Hasselblad 22-Uncoated</ID>
 		<CFA width="2" height="2">

@kmilos
Copy link
Collaborator

kmilos commented Jan 26, 2024

@LebedevRI Maybe add a "Hasselblad: Both 3FR and FFF" note on RPU front page as well ?

@kmilos
Copy link
Collaborator

kmilos commented Jan 26, 2024

Btw, looks like that older Hasselblad - Hasselblad CFV-50c/200 - 16bit (4:3).3FR sample might be corrupted? The conversion to DNG works although...

@LebedevRI
Copy link
Member

Btw, looks like that older Hasselblad - Hasselblad CFV-50c/200 - 16bit (4:3).3FR sample might be corrupted? The conversion to DNG works although...

Yes, it doesn't decode as uncompressed either. I've deleted it.

LebedevRI added a commit to pixlsus/raw that referenced this pull request Jan 26, 2024
@LebedevRI
Copy link
Member

@LebedevRI Maybe add a "Hasselblad: Both 3FR and FFF" note on RPU front page as well ?

pixlsus/raw@858e835

@kmilos
Copy link
Collaborator

kmilos commented Jan 26, 2024

Btw, looks like that older Hasselblad - Hasselblad CFV-50c/200 - 16bit (4:3).3FR sample might be corrupted? The conversion to DNG works although...

Yes, it doesn't decode as uncompressed either. I've deleted it.

Hm, didn't have access to the uncompressed one... The ljpeg failure could be perhaps also related to #144?

I was getting this:

ERROR: [rawspeed] C:/msys64/home/kmilos/rawspeed/src/librawspeed/decoders/RawDecoder.cpp:337: rawspeed::RawImage rawspeed::RawDecoder::decodeRaw(): C:/msys64/home/kmilos/rawspeed/src/librawspeed/io/Buffer.h:80: rawspeed::Buffer rawspeed::Buffer::getSubView(size_type, size_type) const: Buffer overflow: image file may be truncated

and I can also reproduce it on H5D-50c 3FR samples from DPR (maybe one more hint this is indeed the same sensor+processor as in CFV-50c).

@mristuccia
Copy link
Contributor Author

mristuccia commented Feb 7, 2024

@kmilos there is an error in the alias: the id must be corrected as indicated here below.

And, by the way, I was able to identify all aliases for the .3FR files generated by the CFV-50c:

<Alias id="Hasselblad CFV-50c">Hasselblad CFV-50c</Alias>  <-- already there, must be corrected as indicated here
<Alias id="CFV-50c/Flash Sync">CFV-50c/Flash Sync</Alias>
<Alias id="CFV-50c/SWC">CFV-50c/SWC</Alias>
<Alias id="CFV-50c/200">CFV-50c/200</Alias>
<Alias id="CFV-50c/500">CFV-50c/500</Alias>
<Alias id="CFV-50c/Schneider">CFV-50c/Schneider</Alias>
<Alias id="CFV-50c/LensCtrl S">CFV-50c/LensCtrl S</Alias>
<Alias id="CFV-50c/Winder CW">CFV-50c/Winder CW</Alias>
<Alias id="CFV-50c/ELD">CFV-50c/ELD</Alias>
<Alias id="CFV-50c/ELX">CFV-50c/ELX</Alias>
<Alias id="CFV-50c/Pinhole">CFV-50c/Pinhole</Alias>

@kmilos if you want you can add all of them as well. This will guarantee the 3FR compatibility in addition to the FFF one.

I have this digital back and I've understood where the information after the "/" is coming from. Those are the different camera systems this back is compatible with and they can be chosen from its settings menu.

I remain sceptic on the fact that editing a 3FR is not intended by design by Hasselblad though.

@LebedevRI
Copy link
Member

Define "editing"? I guess the question is, what modifications are done to the decoded 3FR before it is re-saved as FFF?
If there aren't any, then there is zero difference of what the user should actually process.

@mristuccia
Copy link
Contributor Author

mristuccia commented Feb 7, 2024

Define "editing"? I guess the question is, what modifications are done to the decoded 3FR before it is re-saved as FFF?
If there aren't any, then there is zero difference of what the user should actually process.

Phocus, the official Hasselblad RAW editor, does not allow to do apply any development to the 3FR file. Only conversion to FFF is possible. So any action that causes the creation of a sidecar .xmp file to the 3FR should be avoided if we want to stick to the original Vendor's intent.

@LebedevRI
Copy link
Member

does not allow to do apply any development to the 3FR file.

That does not, at all, mean that no changes are done to the data.
They, e.g., could be forcibly applying flat field correction, or some bad pixel removal.

So any action that causes the creation of a sidecar .xmp file to the 3FR should be avoided if we want to stick to the original Vendor's intent.

Well, that is rather, err, . The vendor's intent is for consumers
to use the software provided by the vendor, so let's not think about vendor intent too much :)

@mristuccia
Copy link
Contributor Author

mristuccia commented Feb 7, 2024

Well, that is rather, err, . The vendor's intent is for consumers
to use the software provided by the vendor, so let's not think about vendor intent too much :)

Anyway, I'm not able to open the 3FR files despite all the alias are there and despite I've double checked the model name directly from the EXIF data. So, there is something in the 3FR files that is going against your desire not to think about the vendor intent too much. 😅

@LebedevRI
Copy link
Member

LebedevRI commented Feb 7, 2024

Anyway, I'm not able to open the 3FR files despite all the alias are there and despite I've double checked the model name directly from the EXIF data.

Which 3FR in particular?

@kmilos
Copy link
Collaborator

kmilos commented Feb 7, 2024

The inability to open 3FR files for these older models is not due to aliases, but entirely due to #144

3FR files from newer models like CFV II 50C and X1D 50C II open just fine.

When converting to FFF, this is (losslessly again) re-compressed to a more compliant stream for one. We don't know if there is any other processing on top.

@mristuccia
Copy link
Contributor Author

Which 3FR in particular?

Any 3FR coming from my CFV-50c (mark I).

The inability to open 3FR files for these older models is not due to aliases, but entirely due to #144

3FR files from newer models like CFV II 50C and X1D 50C II open just fine.

When converting to FFF, this is (losslessly again) re-compressed to a more compliant stream for one. We don't know if there is any other processing on top.

The model name in the alias is correct, so most probably it is the #144.

For now I'll stick with the process of generating the FFF file by importing the 3FR in Phocus and then I'll develop the FFF file on Darkroom.

@LebedevRI
Copy link
Member

LebedevRI commented Feb 7, 2024

@mristuccia Could you please one of the problematic(!!!) raws,
convert it to FFF, and contribute BOTH of the files to RPU please?

Bonus points if the sample is well-lit (low-iso daylight landscape) and is horizontal.
(Extra bonus points if said raw is fresh, just from the camera, via a good card+cable)

@mristuccia
Copy link
Contributor Author

@mristuccia Could you please one of the problematic(!!!) raws, convert it to FFF, and contribute BOTH of the files to RPU please?

Bonus points if the sample is well-lit (low-iso daylight landscape) and is horizontal. (Extra bonus points if said raw is fresh, just from the camera, via a good card+cable)

Done.

  • Original 3FR straight from the camera card:
20200215_BERLIN_OldNew_WestHafen_B0000420.3FR
  • Corresponding imported FFF file from Hasselblad Phocus software:
20200215_BERLIN_OldNew_WestHafen_B0000420.FFF
  • The 3FR file model name is "CFV-50c/Flash Sync" (that was my setting in the camera when I've shot it).

  • The FFF file model name "Hasselblad CFV-50c" as expected. (Please correct the alias of your merge request as indicated above, otherwise the FFF file won't open correctly in Darktable).

Now, where are my bonus points? 😀

@mristuccia
Copy link
Contributor Author

P.S.
I've spotted a blank character in front of the model name in the exif data of the 3FR file.
It is like " CFV-50c/Flash Sync" rather than "CFV-50c/Flash Sync".

But even including such leading blank character in the alias it does not work.

@LebedevRI
Copy link
Member

Now, where are my bonus points? 😀

Alright, something like this might suffice, perhaps? :)

image

LebedevRI added a commit to LebedevRI/rawspeed that referenced this pull request Feb 8, 2024
LebedevRI added a commit to LebedevRI/rawspeed that referenced this pull request Feb 8, 2024
@LebedevRI
Copy link
Member

#649

LebedevRI added a commit to LebedevRI/rawspeed that referenced this pull request Feb 8, 2024
LebedevRI added a commit to LebedevRI/rawspeed that referenced this pull request Feb 8, 2024
@kmilos
Copy link
Collaborator

kmilos commented Feb 8, 2024

I think we want to remove the part after the / in the id everywhere so this always shows up just as "CFV-50c" in dt. Phocus drops this as well when converting to FFF.

@mristuccia
Copy link
Contributor Author

mristuccia commented Feb 8, 2024

Now, where are my bonus points? 😀

Alright, something like this might suffice, perhaps? :)

image

Well, I think that's enough! 😊
Thank you.

@kmilos
Copy link
Collaborator

kmilos commented Feb 8, 2024

Here's some more bonus points 😉

I've dumped the raw values from both 3FR and FFF using LibRaw's unprocessed_raw tool, then you can load into Python/Matlab/Octave and do a (3FR - FFF) diff. They're not identical, and the curious thing is that when you take look at the normalized difference to 0-1.0 (subtract min, then divide by max-min range), looks like the image structure:

image

And here's normalized division (3FR / FFF):

image

So yeah, looks like there might be some extra processing going on, and it looks like only half of the pixels in the CFA pattern are affected - this is a 4x4 patch from the difference at 100,100:

   -119      0   -120      0
      0    -57      0    -57
   -121      0   -115      0
      0    -57      0    -57

and the corresponding gain patch:

    0.9775    1.0000    0.9774    1.0000
    1.0000    0.9951    1.0000    0.9951
    0.9775    1.0000    0.9775    1.0000
    1.0000    0.9952    1.0000    0.9952

As only R and B channels seem to be affected, this makes me think about the equivalent of the non-identity CameraCallibration matrix in the DNG spec, only that one is global, this one seems to depend on the image content...

@kmilos
Copy link
Collaborator

kmilos commented Feb 8, 2024

And I really think we want #650

@kmilos
Copy link
Collaborator

kmilos commented Feb 8, 2024

Ah, and if I account for the black level before doing 3FR / FFF, it gets more uniform:

image

@mristuccia
Copy link
Contributor Author

mristuccia commented Feb 8, 2024

Here's some more bonus points 😉

I've dumped the raw values from both 3FR and FFF using LibRaw's unprocessed_raw tool, then you can load into Python/Matlab/Octave and do a (3FR - FFF) diff. They're not identical, and the curious thing is that when you take look at the normalized difference to 0-1.0 (subtract min, then divide by max-min range), looks like the image structure:

image And here's normalized division (3FR / FFF): image So yeah, looks like there might be some extra processing going on, and it looks like only half of the pixels in the CFA pattern are affected - this is a 4x4 patch from the difference at 100,100:
   -119      0   -120      0
      0    -57      0    -57
   -121      0   -115      0
      0    -57      0    -57

and the corresponding gain patch:

    0.9775    1.0000    0.9774    1.0000
    1.0000    0.9951    1.0000    0.9951
    0.9775    1.0000    0.9775    1.0000
    1.0000    0.9952    1.0000    0.9952

As only R and B channels seem to be affected, this makes me think about the equivalent of the non-identity CameraCallibration matrix in the DNG spec, only that one is global, this one seems to depend on the image content...

What I can say is that Hasselblad applies a special process to RAW files for getting what they call HNCS (Hasselblad Natural Color Solution) which is based on sensor calibration data (each sensor, and I mean each sensor unit sold, has its own dedicated calibration done at the fabric before selling that specific camera) in conjunction with a special camera profile.
"Hasselblad implemented a colour lookup table which is inspired by film and which is normalized beforehand on a per sensor basis with a gain file per camera; there are multiple LUTs which are intepolated based on Kelvin (two illiuminants, essentially)." [cit.]
The goal of all this is obtaining a very natural color rendition, especially for skin tones. Moreover, the very same rendition is guaranteed for all Hasselblad digital cameras with all kind of sensors they use in the different models.

Maybe this 3FR to FFF difference may be justified by the application of this "gain file" form of normalisation. I don't know. But if it is so, that's another reason why we should work with the "imported" FFF file instead of the original 3FR. That's for sure what I will do for the time being.

@kmilos
Copy link
Collaborator

kmilos commented Feb 8, 2024

Maybe this 3FR to FFF difference may be justified by the application of this "gain file" form of normalisation. I don't know. But if it is so, that's another reason why we should work with the "imported" FFF file instead of the original 3FR. That's for sure what I will do for the time being.

Sure, makes sense if you're looking for ultimate color accuracy as Hasselblad defines it.

But if it really is just a R/B calibration, one should be able to easily corrected for in post in dt, so no reason not to try and enable 3FR support in dt and let people get creative either way!

@mristuccia
Copy link
Contributor Author

Maybe this 3FR to FFF difference may be justified by the application of this "gain file" form of normalisation. I don't know. But if it is so, that's another reason why we should work with the "imported" FFF file instead of the original 3FR. That's for sure what I will do for the time being.

Sure, makes sense if you're looking for ultimate color accuracy as Hasselblad defines it.

But if it really is just a R/B calibration, one should be able to easily corrected for in post in dt, so no reason not to try and enable 3FR support in dt and let people get creative either way!

Sure, to each his own. Freedom is freedom! 😊

@LebedevRI
Copy link
Member

I've dumped the raw values from both 3FR and FFF using LibRaw's unprocessed_raw tool, then you can load into

FWIW, rstest tool has -d option.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Resolved
Development

Successfully merging this pull request may close these issues.

3 participants