pySIPFENN Tests

Core pySIPFENN Functionalities

class pysipfenn.tests.test_pysipfenn.TestCore(methodName='runTest')

Bases: TestCase

Test the core functionality of the Calculator object and other high-level API functions. It does not test the correctness of the descriptor generation functions or models, as these are delegated to other tests.

detectModels()

Test that the updateModelAvailability() method works without errors and returns a list of available models.

setUp()

Initialise the Calculator object for testing. It will be used in all tests and is not modified in any way by them.

testDownloadAndLoadModels()

Tests that the downloadModels() method works without errors in a case whwere the models are not already downloaded and loads them correctly using the loadModels() method. Then also load a model explicitly using loadModel() and check that it is in the loadedModels list. Also check that error is raised correctly if a non-available model is requested to be loaded.

testFromPOSCAR_KS2022()

Update the list of available models and identifies which models are compatible with the KS2022 descriptor. Then it runs featurization from the exampleInputFiles directory. It also tests the printout of the Calculator object after the prediction run.

testFromPOSCAR_Ward2017()

Update the list of available models and identifies which models are compatible with the Ward2017 descriptor. Then it runs featurization from the exampleInputFiles directory.

testFromStructure_KS2022_dilute()

Update the list of available models and identifies which models are compatible with the KS2022_dilute featurization (KS2022 descriptor). Then it runs featurization from the exampleInputFiles directory. It also then checks that the ‘pure’ convenience magic works correctly by comparing the results to the original pure structure results.

testInit()

Test that the Calculator object is initialized correctly.

test_CalculatorPrint()

Test that the Calculator.__str__() method returns the correctly formatted string after being initialized but before predictions.

test_RunModels_Errors()

Test that the runModels() and runModels_dilute() methods raise errors correctly when it is called with no models to run or with a descriptor handling that has not been implemented.

test_WriteDescriptorDataToCSV()

Test that the writeDescriptorsToCSV() method writes the correct data to a CSV file and that the file is consistent with the reference output. It does that with both anonymous structures it enumerates and labeled structures based on the c.inputFileNames list.

test_descriptorCalculate_KS2022_parallel()

Test succesful execution of the descriptorCalculate() method with KS2022 in parallel. A separate test for calculation accuracy is done in test_KS2022.py.

test_descriptorCalculate_KS2022_serial()

Test succesful execution of the descriptorCalculate() method with KS2022 in series. A separate test for calculation accuracy is done in test_KS2022.py.

test_descriptorCalculate_Ward2017_parallel()

Test succesful execution of the descriptorCalculate() method with Ward2017 in parallel. A separate test for calculation accuracy is done in test_Ward2017.py.

test_descriptorCalculate_Ward2017_serial()

Test succesful execution of the descriptorCalculate() method with Ward2017 in series. A separate test for calculation accuracy is done in test_Ward2017.py.

KS2022 Featurization Correctness

class pysipfenn.tests.test_KS2022.TestKS2022(methodName='runTest')

Bases: TestCase

Tests the correctness of the KS2022 descriptor generation function by comparing the results to the reference data for the first 25 structures in the exampleInputFiles directory, stored in the exampleInputFilesDescriptorTable.csv. That file that is also used to test the correctness of the Ward2017, which is a superset of the KS2022.

setUp()

Reads the reference data from the exampleInputFilesDescriptorTable.csv file and the labels from the first row of that file. Then it reads the first 25 structures from the exampleInputFiles directory and generates the descriptors for them. The results are stored in the functionOutput list. It defines the emptyLabelsIndx list that contains the indices of the labels that are not used in the KS2022 (vs Ward2017) descriptor generation. It also persists the test results in the KS2022_TestResult.csv file.

test_resutls()

Compares the results of the KS2022 descriptor generation function to the reference data on a field-by-field basis by calculating the relative difference between the two and requiring it to be less than 1% for all fields except 0-valued fields, where the absolute difference is required to be less than 1e-6.

class pysipfenn.tests.test_KS2022.TestKS2022Profiling(methodName='runTest')

Bases: TestCase

Test the KS2022 descriptor generation by profiling the execution time of the descriptor generation function for two example structures (JVASP-10001 and diluteNiAlloy).

test_parallel()

Test the parallel execution of the descriptor generation function 24 times for each of the two examples but in parallel with up to 8 workers to speed up the execution.

test_serial()

Test the serial execution of the descriptor generation function 4 times for each of the two examples.

KS2022 Dilute-Optimized Featurization Correctness

class pysipfenn.tests.test_KS2022_dilute.TestKS2022(methodName='runTest')

Bases: TestCase

Test the KS2022 descriptor calculator optimized for dilute systems.

setUp()

Import the lables expected for the KS2022 dilute descriptor (same as KS2022) and initialize 4 test materials (mp-13, mp-27, mp-165, mp-1211280) to be used in the tests. The 4 test cases should be sufficient to test the dilute descriptor as general KS2022 is tested more extensively, and problems should propagate to the dilute featurizer. To create the dilute structures, 2x2x2 supercells of the test materials are created and the atom at site 0 is replaced with aluminum. Results for the first test case, comparing general KS2022, explicit base, and implicit (pure) base, are persisted in the KS2022_dilute_TestReslt.csv

test_resutls_assumePure()

Compare the KS2022_dilute featurizer results with general KS2022 assuming the base structure is pure.

test_resutls_explicitBase()

Compare the KS2022_dilute featurizer results with general KS2022 using explicit base structures, i.e. structures from before the dilute element was added. Calculates the relative difference between the two and requires it to be less than 1% for all fields except 0-valued fields, where the absolute difference is required to be less than 1e-6.

class pysipfenn.tests.test_KS2022_dilute.TestKS2022_diluteProfiling(methodName='runTest')

Bases: TestCase

Test the dilute version of KS2022 descriptor generation by profiling the execution time of the descriptor generation function for one example dilute structure.

test_parallel()

Test the parallel execution of the descriptor generation function 64 times but in parallel with up to 8 workers to speed up the execution.

test_serial()

Test the serial execution of the descriptor generation function 10 times.

Ward2017 Featurization Correctness

class pysipfenn.tests.test_Ward2017.TestWard2017(methodName='runTest')

Bases: TestCase

Tests the correctness of the KS2022 descriptor generation function by comparing the results to the reference data for the first 5 structures in the exampleInputFiles directory, stored in the exampleInputFilesDescriptorTable.csv.

setUp()

Reads the reference data from the exampleInputFilesDescriptorTable.csv file and the labels from the first row of that file. Then it reads the first 5 structures from the exampleInputFiles directory and generates the descriptors for them. The results are stored in the functionOutput list. It also persists the test results in the Ward2017_TestResult.csv file.

test_resutls()

Compares the results of the Ward2017 descriptor generation function to the reference data on a field-by-field basis by requiring the absolute difference to be less than 1e-6.

class pysipfenn.tests.test_Ward2017.TestWard2017Profiling(methodName='runTest')

Bases: TestCase

Test the Ward2017 descriptor generation by profiling the execution time of the descriptor generation function for two example structures (JVASP-10001 and diluteNiAlloy).

test_parallel()

Test the parallel execution of the descriptor generation function 24 times for each of the two examples but in parallel with up to 8 workers to speed up the execution.

test_serial()

Test the serial execution of the descriptor generation function 4 times for each of the two examples.

Auto Runtime of All ONNX Models with Ward2017

class pysipfenn.tests.test_AllCompatibleONNX_Ward2017.TestAllCompatibleONNX_Ward2017(methodName='runTest')

Bases: TestCase

_Requires the models to be downloaded first._ It then tests the runtime of the pySIPFENN on all POSCAR files in the exampleInputFiles directory and persistence of the results in a CSV file.

test_runtime()

Runs the test.

Accuracy of NN9 20 24 Predictions Against Reference

class pysipfenn.tests.test_Krajewski2020_NN9NN20NN24_ONNX.TestKrajewski2020ModelsFromONNX(methodName='runTest')

Bases: TestCase

_Requires the NN9/20/24 models to be downloaded first._ It takes the 0-Cr8Fe18Ni4.POSCAR file from the exampleInputFiles directory and calculates the energy with the NN9/20/24 models. The results are then compared to the reference results obtained by authors using pySIPFENN (MxNet->ONNX->PyTorch) and SIPFENN (directly in MxNet) to the 6th decimal place (0.001 meV/atom).

test_resutls()

Runs the test.

Model Exporters Runtime

class pysipfenn.tests.test_ModelExporters.TestExporters(methodName='runTest')

Bases: TestCase

Test all model exporting features that can operate on the Calculator object. Note that this will require the models to be downloaded and the environment variable MODELS_FETCHED to be set to true if running in GitHub Actions.

setUp()

Initialise the Calculator object for testing.

testCoreMLExport()

Test that the CoreML export works with all models with no errors. Please note that if you are using custom descriptors, you will need to add them to the exporter definition in pysipfenn/core/modelExporters.py.

testExceptions1()

Test that the exceptions are raised correctly by the exporters when the Calculator is empty. Regardless of the model presence, it will skip the automatic loading of models to pretend it is a fresh install.

testExceptions2()

Test that the exceptions are raised correctly by the exporters when the models are loaded, but the descriptor they are trying to use is not defined in the exporter.

testInit()

Test that the Calculator object is initialised correctly.

testModelsLoaded()

Test that the models are loaded correctly.

testONNXExport()

Test that the ONNX export works with all models with no errors. For two of the models, the export will also simplify or convert to FP16 to check that it gets correctly encoded in the exported file name.

testONNXFP16()

Test that the ONNX FP16 conversion works with all models with no errors.

testONNXSimplify()

Test that the ONNX simplification works with all models with no errors.

testTorchExport()

Test that the PyTorch export works with all models with no errors. Please note that if you are using custom descriptors, you will need to add them to the exporter definition in pysipfenn/core/modelExporters.py.

Defining Custom Models

class pysipfenn.tests.test_customModel.TestCustomModel(methodName='runTest')

Bases: TestCase

_Requires the models to be downloaded first._ Test loading a custom model by copying the Krajewski2020_NN24 model to the current directory and loading it from there instead of the default location.

setUp()

Copies the model to CWD.

Return type:

None

tearDown()

Deletes the copied model.

Return type:

None

testCalculation()

Loads the model as custom, runs the calculation and compares the results to the original model results field by field.