Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FENDL-3.2b Retrofitting #42

Draft
wants to merge 59 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
dbef6ca
First commit for FENDL3.2B retrofitting
Jun 14, 2024
cdd7bcd
Replacing NJOY Bash script with subprocess execution in Python
Jun 17, 2024
f3d010f
Simplifying mass number formatting in tendl_download() function
Jun 17, 2024
6c5ac24
Simplifying endf_specs() function
Jun 17, 2024
03e3af3
Remove now-obsolete ENDFtk warning suppression
Jun 17, 2024
fa4f29e
Simplify tendl_download() function using data structures
Jun 17, 2024
0190096
Switching tendl_download() function over to urllib dependence
Jun 17, 2024
413ae46
Moving card deck formatting from Pandas DataFrame to dictionary
Jun 17, 2024
2eb9ffd
Separating out a write function for the GROUPR input from the input c…
Jun 17, 2024
1247db3
Removing now-obsolete Pandas dependence
Jun 17, 2024
1d35b79
Simplifying card writing for groupr_input_file_writer()
eitan-weinstein Jun 18, 2024
de3cbb4
Fixing indexing on groupr_input_file_writer()
Jun 18, 2024
d20eed8
Storing elements in a single dictionary to be referenced across both …
Jun 18, 2024
58a4ede
Removing now-obsolete ENDFtk warning supression from gend_tools.py an…
Jun 18, 2024
b83be55
Updating gendf_download() function -- notably switching away from wge…
Jun 18, 2024
e135311
Switching CSV reading from Pandas DataFrame to dictionary
Jun 18, 2024
29528dd
Moving away from direct input to argparse input/options
Jun 18, 2024
0abb51b
Expanding argparse usage
Jun 18, 2024
582424b
Moving away from print statements towards logging
Jun 18, 2024
77e9a65
Removed unnecessary file from file cleanup list
Jun 19, 2024
493a35c
Expanding logger to capture 'No bottleneck testing available' message
Jun 19, 2024
a29bd66
Improving readability of NJOY run message for logger
Jun 19, 2024
69fe5f0
Updating the logging to redirect ENDFtk messages to the logger and re…
Jun 21, 2024
d0f7d3b
Removing stand-alone groupr script -- unnecessary and not called indi…
Jun 21, 2024
4318ae4
Reorganizing folder structure -- separate GROUPR folder no longer see…
Jun 21, 2024
b1b63f9
Finalizing move out of GROUPR/
Jun 21, 2024
1edd251
Moving the rest of fendl3_gendf.py to the main() function
Jun 21, 2024
fb2d548
Forgot to include mt_table in main()
Jun 21, 2024
4065d00
Streamlining endf_specs usage and placement.
Jun 24, 2024
4250c44
Removing direct GENDF download function -- all downloads need to be p…
Jun 24, 2024
93d469f
Moving GROUPR parameters to global constants.
Jun 24, 2024
98dcc93
Logging error if NJOY run is unsuccessful.
Jun 24, 2024
2460d72
Cleaning up package imports
Jun 24, 2024
6fcf5e5
Removing unnecessary package imports on fendl3_gendf.py
Jun 24, 2024
5ec6bbf
Fixing KZA formatting.
Jun 26, 2024
f490d38
Addressing low-level comments from most recent review.
Jul 1, 2024
45df27f
Improving readability
Jul 1, 2024
b76634f
Beginning high-level overhaul and restructuring
Jul 1, 2024
121e57a
Improving readability for nuclear_decay()
Jul 1, 2024
fb1b796
Increasing readability of argparser
Jul 1, 2024
c8e6cea
Major overhaul of modularity and including functionality for iteratin…
Jul 9, 2024
4d99f41
Removing time package.
Jul 9, 2024
e0529dc
Removing specific example file from GENDF files.
Jul 9, 2024
cc064b6
Making the file saving more versatile.
Jul 9, 2024
14c5730
Responding to a majority of the high-level comments from Tuesday's re…
Jul 11, 2024
95815b2
Fixing docstring for ensure_gendf_markers() function.
Jul 11, 2024
a5997b5
Improving isotope identification methods.
Jul 12, 2024
f83a646
Improving isotope identification methods.
Jul 12, 2024
98f23c3
Simplifying logging method and usage.
Jul 12, 2024
498c824
One more logging fix.
Jul 12, 2024
a807f1e
Completing response to last review and making arg processing more mod…
Jul 16, 2024
76b9aa1
Improving ability to iterate over all elements.
Jul 16, 2024
eebeea3
Fixing minor bug in execution of handle_TENDL_downloads().
Jul 16, 2024
912530f
Small formatting change to fit in max line length.
Jul 16, 2024
c374494
More minor formatting adjustments and simplifying the line length set…
Jul 16, 2024
f50b617
Allowing for fendle_retrofit.py to be executed from DataLib.
Jul 16, 2024
4ba725e
Removing unnecessary print statement.
Jul 17, 2024
6257033
Ensuring that NJOY output is properly handled when program is execute…
Jul 17, 2024
4921dba
Small formatting changes before moving over to project individual PRs.
Jul 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions src/DataLib/fendl32B_retrofit/fendl3_gendf.py
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separation of concerns - make this more modular with each function performing a specific task.

  • argparse for command-line input/options
  • read JSON or YAML for more complicated input
  • separate methods for reading data, processing data, outputting data
  • logging for user output

Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Import packages
import ENDFtk
import gendf_tools as GENDFtk
import pandas as pd
import sys
import subprocess
from logging_config import logger
import groupr_tools as GRPRtk

# Load MT table
# Data for MT table collected from
# https://www.oecd-nea.org/dbdata/data/manual-endf/endf102_MT.pdf
mt_table = GENDFtk.read_csv('mt_table.csv')

def main():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This main method is doing too many things and I don't know why. You probably need some more modularity, with each function having a docstring to clarify what it's purpose is


# Parse command line arguments
args = GENDFtk.fendl_args()

# Set conditionals for local file input
if args.method == 'I':
gendf_path = args.local_path
pKZA = GENDFtk.gendf_pkza_extract(gendf_path)

# Set conditionals for file download
elif args.method == 'D':
element = args.element
A = args.A
# Check isomeric state
if 'm' not in A:
gendf_path, pKZA = GENDFtk.gendf_download(element, A)
else:
# Use NJOY GROUPR to convert the isomer's TENDL 2017 data to a GENDF file

# Download ENDF and PENDF files for the isomer
endf_path = GRPRtk.tendl_download(element, A, 'endf')
pendf_path = GRPRtk.tendl_download(element, A, 'pendf')

# Extract necessary MT and MAT data from the ENDF file
matb, MTs = GRPRtk.endf_specs(endf_path)

# Write out the GROUPR input file
card_deck = GRPRtk.groupr_input_file_format(matb, MTs, element, A, mt_table)
GRPRtk.groupr_input_file_writer(card_deck, MTs)

# Run NJOY with GROUPR to create a GENDF file for the isomer
gendf_path = GRPRtk.run_njoy(card_deck, element, A)

# Save pKZA value
pKZA = GENDFtk.gendf_pkza_extract(gendf_path, M = 1)

# Clean up repository from unnecessary intermediate files from GROUPR run
groupr_files = ['groupr.inp', 'groupr.out', 'tape20', 'tape21', 'tape31']
for file in groupr_files:
subprocess.run(['rm', file])

logger.info(f"GENDF file path: {gendf_path}")
logger.info(f"Parent KZA (pKZA): {pKZA}")

# Read in data with ENDFtk
with GRPRtk.redirect_ENDFtk_output():
tape = ENDFtk.tree.Tape.from_file(gendf_path)
mat_ids = tape.material_numbers
mat = mat_ids[0]
xs_MF = 3
file = tape.material(mat).file(xs_MF)

# Extract the MT numbers that are present in the file
MTs = [MT.MT for MT in file.sections.to_list()]

# Initialize lists
cross_sections_by_MT = []
emitted_particles_list = []
dKZAs = []

# Extract data for each MT
for MT in MTs:
try:
sigma_list = GENDFtk.extract_cross_sections(file, MT)
if not sigma_list:
continue
dKZA, emitted_particles = GENDFtk.reaction_calculator(MT, mt_table, pKZA)
if dKZA is None:
continue
cross_sections_by_MT.append(sigma_list)
dKZAs.append(dKZA)
emitted_particles_list.append(emitted_particles)
except Exception as e:
logger.error(f"Error processing MT {MT}: {e}")
continue

# Store data in a Pandas DataFrame
gendf_data = pd.DataFrame({
'Parent KZA': [pKZA] * len(dKZAs),
'Daughter KZA': dKZAs,
'Emitted Particles': emitted_particles_list,
'Cross Sections': cross_sections_by_MT
})

# Save to CSV
gendf_data.to_csv('gendf_data.csv', index=False)
logger.info("Saved gendf_data.csv")
logger.info(gendf_data.head())

# Execute main() function based on arguments
if __name__ == '__main__':
main()
172 changes: 172 additions & 0 deletions src/DataLib/fendl32B_retrofit/gendf_tools.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# Import packages
import argparse
import csv
import requests
import sys
from groupr_tools import elements
from logging_config import logger

# Define an argument parser
def fendl_args():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's most common to put the argument parser in the file that has the main method. Is there a reason it's here?

# Temporarily reset stdout and stderr to the defaults
# so that arg messages (i.e. --help) print out to terminal
original_stdout = sys.stdout
original_stderr = sys.stderr

try:
sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need this? It seems pretty unusual for an argparse setup. I've never seen this before

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried commenting this out, but it had the effect of sending the argparse help messages to the logger, rather than the terminal, which seems less helpful or desirable than having it print to the terminal, which it currently does.


parser = argparse.ArgumentParser()

# Subparsers for 'I' and 'D'
subparsers = parser.add_subparsers(dest='method', required=True)
parser_I = subparsers.add_parser('I', help='Local file input')
parser_I.add_argument('--local-path',
required=True,
help='Path to the local GENDF file.')
parser_D = subparsers.add_parser('D', help='Download GENDF file')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This downloads tendl/pendf files, not GENDF

parser_D.add_argument('--element', '-e',
required=True,
help= 'Chemical symbol for selected element (i.e. Ti).')
parser_D.add_argument('--A', '-a',
required=True,
help='Mass number for selected isotope (i.e. 48). If the target is an isomer, "m" after the mass number (i.e. 48m)')

args = parser.parse_args()
return args

finally:
# Restore stdout and stderr to the logger
sys.stdout = original_stdout
sys.stderr = original_stderr

# Define a function to read CSV files
def read_csv(csv_path):

# Initialize an empty dictionary
data_dict = {}

# Open the CSV file
with open(csv_path, mode='r') as file:
# Create a CSV reader object
csv_reader = csv.DictReader(file)

# Initialize the dictionary keys with empty lists
for column in csv_reader.fieldnames:
data_dict[column] = []

# Populate the dictionary with the CSV data
for row in csv_reader:
for column in csv_reader.fieldnames:
data_dict[column].append(row[column])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't need to happen inside the with

Copy link
Member

@gonuke gonuke Jun 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is the best data structure for this data. When I look at how you use it, it seems you'd rather have a dictionary with key of the MT number and data of the other columns, esp. the 2nd column.

Data structure choice is important in python for a few reasons:

  1. it prevents fragile data design - this one is theoretically fragile, but probably fine the way it is used
  2. it allows for more readable code.

I think you want to translate these rows into a single dict with something like:

for row in csv_reader:
    data_dict[row['MT']] = row['reaction']

return data_dict

# Define a function to download the GENDF file from nds.iaea.org
def gendf_download(element, A, M=None, save_path=None):
# Initialize parameters
Z = str(elements[element]).zfill(2)
A = str(A).zfill(3)
gendf_gen_url = 'https://www-nds.iaea.org/fendl/data/neutron/group/'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! These GENDF files are NOT activation files. They are transport files and not reliable for our activation purposes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll hold off reviewing the GENDF stuff until we are sure it's useful

download_url = f'{gendf_gen_url}{Z}{element}_{A}.g'
save_path = save_path or f'./fendl3_{element}{A}'

# Check to see if the URL is valid
response = requests.head(download_url)
if response.status_code == 404:
raise FileNotFoundError(f'{download_url} not found')
elif response.status_code == 301:
download_url = f'{gendf_gen_url}{Z}{element}{A}.g'

# Download the file from the URL
logger.info(f'Downloading file from: {download_url}')
response = requests.get(download_url, stream=True)
with open(save_path, 'w') as f:
f.write(response.content.decode('utf-8'))
logger.info(f'Downloaded file saved to: {save_path}')

# Write out the pKZA
M = M or '0'
pKZA = int(Z + A + M)

return save_path, pKZA

# Extract pKZA data from a GENDF file
def gendf_pkza_extract(gendf_path, M=None):
with open(gendf_path, 'r') as f:
first_line = f.readline()
logger.info(f"First line of GENDF file: {first_line}")
Z, element, A = first_line.split('-')[:3]
A = A.split(' ')[0]
if 'm' in A:
m_index = A.find('m')
A = A[:m_index]
M = str(str(M).count('m')) or '0'
pKZA = int(Z + A + M)
return pKZA

# Extract cross-section data for a given MT
def extract_cross_sections(file, MT):
try:
section = file.section(MT).content
lines = section.split('\n')[2:-2:2]
sigma_list = []
for line in lines:
sigma = line.split(' ')[2]
sign = 1 if '+' in sigma else -1
mantissa, exponent = sigma.split('+') if sign == 1 else sigma.split('-')
sigma_list.append(float(mantissa) * (10 ** (sign * int(exponent))))
return sigma_list
except Exception as e:
logger.error(f"Error extracting cross sections for MT {MT}: {e}")
return []

# Count emitted particles
def emitted_particle_count(particle, emitted_particle_string):
particle_index = emitted_particle_string.find(particle)
number_str = ''
for i in range(particle_index - 1, -1, -1):
if emitted_particle_string[i].isdigit():
number_str = emitted_particle_string[i] + number_str
else:
break
return int(number_str) if number_str else 1 if particle in emitted_particle_string else None

# Check for isomers
def isomer_check(emitted_particle_string):
last_digits_str = ''
for char in reversed(emitted_particle_string):
if char.isdigit():
last_digits_str = char + last_digits_str
else:
break
return int(last_digits_str) if last_digits_str else 0

# Calculate reaction
def reaction_calculator(MT, mt_table, pKZA):
try:
nucleus_protons = int(str(pKZA)[:2])
A = int(str(pKZA)[2:5])
index = mt_table['MT'].index(str(MT))
reaction = mt_table['Reaction'][index]
emitted_particles = reaction.split(',')[1][:-1]
particle_types = ['n', 'd', 'alpha', 'p', 't', '3He']
emission_tuples = [(emitted_particle_count(p, emitted_particles), p) for p in particle_types if emitted_particle_count(p, emitted_particles)]

nucleus_neutrons = A - nucleus_protons + 1
for num_particle, particle in emission_tuples:
if particle == 'n':
nucleus_neutrons -= num_particle
if particle == 'p':
nucleus_protons -= num_particle

residual_A = str(nucleus_neutrons + nucleus_protons).zfill(3)
M = isomer_check(emitted_particles)
if M != 0:
emitted_particles = emitted_particles[:-len(str(M))]

dKZA = int(f"{str(nucleus_protons)}{residual_A}{str(M)}")
return dKZA, emitted_particles
except Exception as e:
logger.error(f"Error in reaction calculation for MT {MT}: {e}")
return None, None
Loading