Data Evaluation

[howard9199]

Using MSE, MAE and correlation to compare with U-Net and OBS.
The Evaluation Function is attached below.

[howard9199]

# using inside and outside to calculate the accuracy by Mean Square Error, (inside - outside)^2 is the MSE
# inside: the list of the inside data
# outside: the list of the outside data
# return: the accuracy of the model
def evaluation(inside, outside,data_name,model_name):
    # iterate the inside and outside data
    print(len(inside),len(outside))
    for i in range(len(inside)):
        # reshape the inside and outside data to 1D array
        inside_single = inside[i].reshape(1, -1)[0]
        outside_single = outside[i].reshape(1, -1)[0]
        # calculate the MSE, MAE, corelation
        # print(MSE(inside_single, outside_single), MAE(inside_single, outside_single), corelation(inside_single, outside_single))
        
        # save to .csv file
        # MSE, MAE, corelation should be in the same table
        # the first column is the data_name, the second column is the MSE, the third column is the MAE, the fourth column is the corelation

        # if the file is not exist, create the file and write the first line
        if not os.path.exists('evaluation.csv'):
            with open('evaluation.csv', 'w') as f:
                f.write('data_name,MSE,MAE,corelation\n')
        # write the data to the file
        with open('evaluation.csv', 'a') as f:
            f.write(data_name[i] + ',' + str(MSE(inside_single, outside_single)) + ',' + str(MAE(inside_single, outside_single)) + ',' + str(corelation(inside_single, outside_single)) + '\n')
    with open('evaluation.csv', 'a') as f:
        inside_mean = np.mean(inside, axis=0).reshape(1, -1)[0]
        outside_mean = np.mean(outside, axis=0).reshape(1, -1)[0]
        print(inside_mean)
        f.write(model_name+'average' + ',' + str(MSE(inside_mean, outside_mean)) + ',' + str(MAE(inside_mean, outside_mean)) + ',' + str(corelation(inside_mean, outside_mean)) + '\n')
    return 0

def MSE(inside, outside):
    # L2 norm
    return np.square(np.subtract(inside, outside)).mean()

def MAE(inside, outside):
    # L1 norm
    return np.abs(np.subtract(inside, outside)).mean()

def corelation(inside, outside):
    return np.corrcoef(inside, outside)[0, 1]

[howard9199]

Result: Compare with U-Net and OBS

	U-Net	OBS
Mean Square Error	0.09	0.25
Mean Absolute Error	0.21	0.35
Correlation	0.74	0.01