CrystalShift: A Versatile Command-Line Tool for Crystallographic Structural Data Analysis, Modification, and Format Conversion Prior to Solid-State DFT Calculations of Organic Crystals

Abstract

CrystalShift is an open-source computational tool tailored for the analysis, transformation, and conversion of crystallographic data, with a particular emphasis on organic crystal structures. It offers a comprehensive suite of features valuable for the computational study of solids: format conversion, crystallographic basis transformation, atomic coordinate editing, and molecular layer analysis. These options are especially valuable for studying the mechanical properties of molecular crystals with potential applications in organic materials science. Written in the C programming language, CrystalShift offers computational efficiency and compatibility with widely used crystallographic formats such as CIF, POSCAR, and XYZ. It provides a command-line interface, enabling seamless integration into research workflows while addressing specific challenges in crystallography, such as handling non-standard file formats and robust error correction. CrystalShift may be applied for both in-depth study of particular crystal structure origins and the high-throughput conversion of crystallographic datasets prior to DFT calculations with periodic boundary conditions using VASP code.

1. Introduction

Crystallographic data analysis and processing is a substantial step in materials science, solid-state physics, and computational chemistry. It helps to obtain various data on the nature of the crystal structure: its stability, mechanical properties depending on applied pressure, temperature changes, etc. [1,2,3,4,5,6,7]. In some cases, it is important to perform numerical experiments using computational tools to overcome experimental challenges, save time, and achieve reproducibility [8,9,10].

The visualization and analysis of crystal structure can be performed using multiple software packages, such as CCDC Mercury [11,12], VESTA [13], Chemcraft [14], and VMD [15]. Some of these packages (e.g., CCDC Mercury [12] and ToposPro [16]) provide internal modules and implemented algorithms for advanced crystallographic and topological analysis, as well as basic molecular mechanics calculations. Others, such as VESTA, are more aimed at crystal structure editing using mainly a graphical user interface (GUI) but lack valuable crystal structure analysis tools. Nevertheless, most of the above-mentioned software packages generate mainly crystallographic files, which cannot be used directly by most computational programs (except the peculiar CrystalExplorer21 [17] and OCC [18]). Thus, several scripts and other tools have been published to overcome chemical file formatting issues, which are different for various computational programs. Among others, OpenBabel [19] and Cif2Cell [20] seem to be the most widespread in the community but have to maintain up-to-date input file formats for different programs, which may result in some inconsistencies. There are also services that are designed for specific tasks, such as mechanical properties calculation and different modulus evaluation—some aim to prepare and simplify routine prior to calculations (e.g., DeformCell [21], VASPKIT [22]), while others aim to analyze obtained data (EoSFit7 [23,24], ELATE [25], etc.). Obviously, there are numerous less popular scripts for the particular conversion of files to exact formats (e.g., cif2vasp and vasp2cif [26]), which usually generate input and output files for DFT calculations. Other complex tools, such as pymatgen (a robust, open-source Python library for materials analysis) [27] and ASE (a set of tools and Python modules for setting up, manipulating, running, visualizing, and analyzing atomistic simulations) [28] may be used for comprehensive computational tasks, including not only file format conversion but also data manipulation and results analysis. Nevertheless, both tools require Python knowledge and have a relatively high entry barrier and steep learning curve, which does not encourage their use by computational chemists for simple workflows or very specific tasks.

On the one hand, some software provides a GUI, which helps to inspect crystal structures visually and provides an in-depth analysis of a particular system, but usually limits the possibility of a high-throughput workflow in terms of further DFT calculations. On the other hand, command-line programs are limited in terms of crystal structure analysis and may be sensitive to installed libraries and operating systems, primarily on the supercomputer. An attempt to provide a seamless workflow for calculating the mechanical properties of “bending” crystals results in the usage of multiple programs, as shown in Scheme 1 below:

Thus, despite the availability of these tools, there remains a need for versatile, high-performance software that integrates advanced crystallographic editing, molecular layer analysis, and efficient format conversion. CrystalShift was developed to address this gap, combining robust command-line functionalities with algorithms tailored for detailed structural analysis. It does not require knowledge of any programming language and may be easily used in various workflows on supercomputers using ordinary Bash scripts. Notably, it includes a molecular layer analysis module that identifies clusters of atoms (molecules), detects crystallographic disorders, and calculates interlayer distances using DBSCAN [29] and KD-Tree [30] algorithms, which is valuable for organic systems. Understanding the geometry of molecular layers and their influence on material characteristics is crucial for crystallographic and topological analysis for advancing materials sciences, especially for bending crystals [31]. While electronic structure ultimately governs a material’s behavior, geometric patterns such as interlayer distances, symmetry, and cluster distributions offer predictive insights into a structure’s properties.

2. Software

2.1. Software Design and Architecture

CrystalShift is a command-line tool written in the C programming language, designed for the efficient processing of crystallographic data. It consists of two subroutines: “crystalshift” and “crystalshift_layers”. The “crystalshift” subroutine is designed for converting formats, crystallographic basis and unit cell change, supercell generation, and atomic coordinate editing directly for CIF files from CSD and can be easily adopted for high-throughput workflow. The “crystalshift_layers” subroutine is designed for molecular layer analysis, which is usually needed for specific in-depth modification of the crystal structure prior to solid-state calculations (Scheme 2).

Key modules of “crystalshift” and “crystalshift_layers” subroutines include several source and header files (Scheme 3). The modular structure of the CrystalShift program allows users to isolate and refine specific components of the software without disrupting the entire system. This flexibility is essential, as it allows developers, or even users themselves, to incorporate new algorithms or data processing methods effortlessly (if needed), thereby enhancing the program’s capabilities and versatility. However, this does not hinder the user experience, as all modules are compiled using a single command or Makefile invocation. For installation instructions and access to the program’s code, please refer to the project page on GitHub (https://github.com/shes73/CrystalShift) (accessed on 29 May 2025).

2.2. Input and Output Formats (Converter)

CrystalShift (currently) supports three widely used crystallographic file formats:

• CIF (Crystallographic Information File) is the most widely used format for crystallographic data. CIF files are parsed to extract lattice parameters, atomic coordinates, and additional data. Experimental error data and non-crystallographic information are stripped for clarity during editing. When writing CIF files, CrystalShift simplifies structures by assuming a triclinic lattice with space group P1, avoiding complications from symmetry operations.
• The POSCAR file format is specific to the Vienna Ab initio Simulation Package (VASP) [32,33,34,35], which is widely used for computational materials science. It represents lattice parameters in the form of a matrix and atomic coordinates. This format is extensively used in Density Functional Theory (DFT) calculations and other atomistic simulations. The output from CrystalShift POSCAR files is written in fractional (direct) coordinates, with support for reordering atomic species based on user-defined criteria (e.g., from lightest to heaviest element, otherwise, or user-defined order).
• The XYZ file format is a simple, human-readable format, used primarily for calculations and visualizations of single molecules. It lists the number of atoms and atomic coordinates only. In the extended version of XYZ, there are additionally added lattice parameters in the comment line. This format is very useful for further calculations using other computational software (ORCA [36], Gaussian [37], etc.).

2.3. Basis Change and Supercell Generation

The basis change module includes the creation of supercells since they differ only in the addition of new atoms. When changing the basis, only the vectors and angles of the crystal lattice change, and the number of atoms remains the same. It is worth noting the limitation in the work of the module for creating supercells—CrystalShift can add new atoms only if the new vector components are integers. CrystalShift handles symmetry operations from the CIF file with a separate command, while this module adds atoms solely based on the principle of translation. Thus, if only a unit cell is provided, it is mandatory to convert it to the primitive cell using the appropriate CrystalShift option and then use basis change or supercell generation.

2.4. Molecular Layer Analysis

The molecular layer analysis subroutine “crystalshift_layers” of CrystalShift is designed to identify and characterize molecular layers within crystal lattices, with a focus on applications in material science, particularly for studying flexible organic crystals. The subroutine employs a combination of advanced algorithms and computational techniques to deliver accurate and insightful results. Two approaches were evaluated for identifying sliding planes during the development of the “crystalshift_layers” module—including the PCA algorithm [38,39,40] and the HKL algorithm (Scheme 4).

In the case of PCA, the analysis begins with identifying molecular clusters within the crystal lattice using the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm [29]. DBSCAN is particularly effective for this purpose, as it groups atoms based on their proximity while treating isolated or “noise” atoms as potential lattice disorders. This ensures a robust differentiation between meaningful molecular clusters and extraneous atomic noise.

To optimize spatial queries, such as finding neighboring atoms and clusters, the module leverages a KD-Tree data structure [30]. This significantly enhances computational efficiency, particularly for large datasets with complex geometries.

All obtained data after parsing are used to calculate the eigenvalues and vectors of the covariance matrix via Principal Component Analysis (PCA).

PCA is a widely used statistical method in data analysis and machine learning. Its primary goal is to reduce the dimensionality of a dataset while retaining as much variability (information) as possible. The first principal component captures the direction of the highest variance in the data, the second principal component captures the next highest variance (orthogonal to the first), and so on. PCA involves calculating the eigenvalues and eigenvectors of the covariance matrix of the data. The eigenvectors determine the directions of the principal components, while the eigenvalues indicate the amount of variance captured by each component [38,39,40]. When applied to atomic coordinates, the eigenvector corresponding to the highest variance is expected to align with the molecular layer direction in the crystal lattice.

However, several limitations were identified when PCA was used:

• Molecular layers should be “sufficiently” spaced apart.
• Errors may occur if the molecules are large and bulky. At best, the direction may be slightly incorrect, and at worst, an erroneous result may be obtained.
• PCA analyzes the spatial distribution of atoms rather than explicitly identifying unoccupied surfaces. Consequently, the obtained Miller indices (hkl) may be suboptimal for further analysis.

If the principal axis identified in the first iteration does not meet the criteria above, the program attempts the next (orthogonal) principal axis.

As an alternative approach, a systematic enumeration of all possible Miller index combinations was implemented, ranging from −7 to +7 for each index (h, k, l). The algorithm iterates through each combination, formulates the corresponding plane equation, and evaluates whether atomic positions align with the plane within a predefined threshold.

This approach also has some disadvantages:

• To correctly check large Miller indices, it is necessary to construct supercells.
• The algorithm is unable to identify molecular layers if they correspond to multiple distinct sets of low Miller indices. For instance, if molecular layers are located on the (200) and (300) planes, the method cannot determine an appropriate index combination to define an intermediate plane between them.

For an enhanced user experience, it is advisable to employ these approaches in conjunction to avoid overlooking potential slip planes. While this tool does not guarantee absolute precision, it serves as an excellent supplementary tool.

The molecular layer analysis module is particularly valuable for studying flexible organic crystals, where layered structures are often key to their mechanical properties, such as plasticity or elasticity. By identifying layers, assessing interlayer distances, and detecting lattice disorders, the tool provides researchers with a deeper understanding of how molecular geometry influences material behavior.

2.5. Programming Requirements

CrystalShift is distributed as open-source software, allowing researchers to compile and run the program on various operating systems. The compilation process requires only a standard C compiler, and the Makefile included in the repository simplifies the build process.

By maintaining a minimal dependency footprint while offering advanced analytical capabilities, CrystalShift ensures accessibility and usability across diverse research environments. This careful balance of simplicity and sophistication underpins its versatility as a tool for crystallographic data manipulation and analysis prior to DFT calculations.

All CrystalShift modules, including basis transformations, atomic coordinate editing, format conversion, and molecular layer searching, rely solely on standard C libraries, ensuring broad compatibility, ease of deployment, and usage.

3. Results

The principal idea of CrystalShift is to help computational chemists process CIF-to-VASP input POSCAR files, with the possibility of additional in-depth modification and analysis without using multiple software tools or programming knowledge.

In order to help process multiple crystallographic data used for specific scientific aims, benchmarking, dataset construction, and other possible tasks, we provide feasible tests for converting file formats, estimating calculation speed, possible inaccuracies and mistakes during various procedures, possible misuse by inexperienced users, and formulating current limitations and future improvements. An example of CrystalShift use for a change in the basis, atomic coordinate shift, and conversion to POSCAR from a CIF file is provided in the Worked Example section and on the GitHub project page.

3.1. Feature Validation and Testing

CrystalShift was tested across its core functionalities to validate accuracy, compatibility, and computational efficiency. Validation included the following:

• A semi-automated inspection of the correctness of the output file structure, as well as of the obtained results after editing the crystallographic data, starting with calculations in VASP (controlling possible errors during input file reading).
• A comparison with results from existing tools (e.g., Open Babel, cif2cell, pymatgen, ASE).
• The testing of converters was carried out automatically using Bash scripts. The conversion of CIF → POSCAR → CIF and CIF → XYZ → CIF was carried out. Due to the fact that atoms are recorded in groups classified by elements in POSCAR files, an additional program was written for sorting and obtaining statistics by comparing coordinates in the original CIF file with those in the CIF file obtained after conversion. In this way, 1000 structures, randomly selected from the CSD, were analyzed and showed a 100% success rate. Speed tests were conducted on 100 random structures from the CSD.
• The testing of the layer analysis module was carried out manually, by comparing visually observed molecular layers, slip planes calculated via CCDC Mercury, and results calculated by CrystalShift.

One can find all the results obtained during feature validation and testing on the project’s GitHub page.

3.2. Error Handling

In the structures, all data (e.g., lengths of lattice vectors, volume, atomic coordinates, etc.) is stored in double float format. The range of doubles is 1.7 × 10⁻³⁰⁸ to 1.7 × 10³⁰⁸. Hence, when a file is converted from the CIF format to POSCAR, no coordinate transformations occur. All experimental measurement errors from CIF files are removed for correct recording into structures and, accordingly, are lost during conversion.

When the basis or recording is changed in the XYZ format (which implies conversion into the Cartesian basis), minor discrepancies may occur in the calculation. To calculate new lattice parameters and angles, a scalar product is used, but to recalculate coordinates, an inverse matrix must be used. The inverse matrix is calculated using the Gauss–Jordan method, and then it is reduced to an upper-triangular form using the Gauss method, which entails some inaccuracy due to the iterative approach. Nevertheless, possible inaccuracies are less than experimental uncertainties in SCXRD experimental data and, thus, do not influence further calculations.

Additionally, CrystalShift autonomously detects irregularities in lattice parameters and atomic coordinates, such as duplicate atoms or inconsistencies in atomic groupings. When such anomalies are identified, the program provides warnings to the user, highlighting potential issues. Along with the warnings, CrystalShift suggests corrective actions to address these irregularities, ensuring data integrity and reliable analysis. This proactive feature minimizes the risk of errors in downstream operations while maintaining user control over data adjustments.

3.3. User Warnings

CrystalShift includes several user-focused warnings and safeguards to enhance reliability and prevent unintended errors during use. The program automatically identifies and removes duplicate atoms in the structure if needed, alerting users to potential issues in their input data.

Users are also strongly advised to carefully review both input and output files to ensure accuracy, as this is a crucial step for any crystallographic or computational tool. In the event of an error, CrystalShift provides clear and detailed explanations of possible issues, helping users to diagnose and correct them prior to further DFT calculations.

These measures contribute to a robust and user-friendly experience, minimizing disruptions and ensuring reliable operation.

3.4. Limitations and Areas for Improvement

CrystalShift, while robust in its core functionalities, has certain limitations that present opportunities for enhancement. The reliance on a command-line interface may pose challenges for users who are less familiar with non-graphical tools, potentially limiting accessibility.

The current version of the program writes POSCAR files in direct coordinates, which restricts compatibility with workflows that require the Cartesian format. Implementing this feature is planned for upcoming updates.

While molecular layer analysis offers valuable insights, expanding the range of geometric analyses to include features like bond length distributions, angular analysis, and void space quantification would significantly enhance its utility and broaden its applicability across diverse research domains.

3.5. Worked Example

To reveal the practical use of CrystalShift, a 4-bromophenyl 4-bromobenzoate structure was selected. Crystals of 4-bromophenyl 4-bromobenzoate demonstrate phenomenal plasticity under mechanical stress, which is explained by the layered structure and significant anisotropy in the crystal structure [41,42,43]. Thus, the calculation of mechanical properties and their correlation with crystal structure seems to provide valuable information for better understanding the nature of the “bending crystals” phenomenon [44,45,46].

A CIF file of 4-bromophenyl 4-bromobenzoate (VEWSIC) was downloaded from CSD and submitted to CrystalShift. “Crystalshift_layers” suggested the (1 0 −1) slip plane as the most favorable. Visual inspection in CCDC Mercury software confirmed this choice (Figure 1).

The straightforward calculation of second-order derivatives, taking into account multiple computational parameters, elucidates the mechanical characteristics along crystallographic axes. Nevertheless, the 4-bromophenyl 4-bromobenzoate crystal structure is constructed of “diagonal” layers, which are not aligned with the a, b, or c crystallographic axis. This peculiarity limits the convenience of the obtained data interpretation. It becomes obvious that a basis change may help to obtain mechanical properties and correlate them to the crystal structure in a straightforward and convenient manner.

To address the task of identifying quantitative criteria for determining the unique mechanical properties of organic crystals, extensive data manipulation is required. The following steps outline the process of editing a file using the example of 4-bromophenyl 4-bromobenzoate within a practical workflow. Thus, the “crystalshift” subroutine was used to change the basis and prepare the input file for DFT calculations in the VASP package (Figure 2).

To verify the accuracy of the aforementioned procedures in CrystalShift, DFT calculations of lattice energies were performed using the PBE functional [49], a plane–wave basis set with a kinetic energy cutoff of 800 eV and projector augmented wave (PAW) atomic pseudopotentials [50,51], using the D3BJ dispersion correction scheme [52]. Monkhorst–Pack k-point meshes [53] of 2 × 5 × 1 and 1 × 6 × 1 were used for the initial and modified structures. The normalized energy for the lattice after the basis change (×2 supercell) differed by only 0.02 kJ/mol, with a minimal 0.049 RMSD for the atoms in the initial and changed cells (Figure 3).

Finally, the calculated mechanical properties (using a finite-difference approach [54]) indicate the presence of anisotropy along the z-axis, which coincides with the c crystallographic axis in the structure after the basis change (Figure 4). The obtained results correspond to the detected molecular layers in the structure. This fully coincides with Reddy’s model, describing bending crystals as layered structures with strong interactions within the layers and weak interactions in nearly perpendicular directions [48]. Moreover, the obtained results support previous data on the calculated mechanical properties of the plastic form of 4-bromophenyl 4-bromobenzoate [42,43].

This example of bending 4-bromophenyl 4-bromobenzoate crystals shows how CrystalShift may help with research preparation and data interpretation more simply and intuitively in comparison to the current workflow.

4. Discussion

The development and implementation of CrystalShift addresses several longstanding challenges in crystallographic data processing, positioning it as a valuable alternative and supplement to existing tools. While programs like Open Babel, Mercury, Chemcraft, VESTA, cif2cell, pymatgen, ASE, etc. offer distinct advantages, CrystalShift fills critical gaps in functionality and versatility, particularly for computational workflows involving molecular crystals. It is written in the C programming language, has minimal dependency requirements, is compatible across multiple computing environments, and does not require programming knowledge from the user.

The molecular layer analysis module of CrystalShift represents a significant advancement over existing tools. By employing DBSCAN and KD-Tree algorithms, this module not only identifies molecular clusters and detects lattice disorders but also calculates interlayer distances and structural patterns. This functionality is particularly valuable for studying bending organic crystals, where understanding geometric layering is essential for predicting mechanical flexibility, thermal stability, and other material properties.

Despite its strengths, CrystalShift does have certain limitations. For example, while the current version supports CIF, POSCAR, and XYZ file formats, additional format compatibility (.mol, .pdb, etc.) could further expand its utility. Future developments could also enhance features such as more comprehensive molecular distribution analysis, ensuring CrystalShift remains competitive in the rapidly evolving field of crystallography.

5. Conclusions

CrystalShift is a command-line tool that can be easily used by computational chemists in order to process and modify original crystallographic data for VASP calculations. It has minimal dependencies and can be installed on almost any supercomputer or workstation.

It contains a basic file converter that can be effectively used for high-throughput VASP calculations of selected structures from the CSD. It is also useful for the in-depth study of molecular crystals and their properties, offering intuitive crystallographic basis change and supercell generation options, as well as a layer analysis module. The latter is of significant importance for bending crystals, which exhibit plastic or elastic behavior.

CrystalShift helps researchers study organic crystals by addressing key technical and practical challenges in crystallographic and computational workflows.