Crystals of the β'-type Pd(dmit)₂ salts belong to a monoclinic system with space group C2/c. The unit cell contains two crystallographically equivalent Pd(dmit)₂ anion layers (1 and 2), which are separated by a cation layer (Figure 2). The Pd(dmit)₂ layers are related by glide plane symmetry. The cations are located on a two-fold axis, and thus cations without two-fold symmetry (EtMe₃Z⁺) may adopt two possible orientations with an occupancy of 50% for each orientation. In the Pd(dmit)₂ layer, Pd(dmit)₂ molecules are strongly dimerized with an eclipsed overlapping mode. The number of crystallographically independent dimer units is one. The two Pd(dmit)₂ molecules in the dimer unit are equivalent and related by inversion symmetry. The dimer units stack along the a + b direction in layer 1 and along the a − b direction in layer 2 (Figure 2; solid-crossing column structure). Table 1 shows the lattice constants of β'-(Et_xMe_4−xZ)[Pd(dmit)₂]₂, plotted as a function of the covalent radius of the central atom Z (R_Z) in Figure 3. Understandably, the effect of the number of ethyl groups (x) is larger than that of R_Z in general. With decreasing R_Z and x, the unit cell volume V decreases. In this sense, the cation effect appears equivalent to an application of pressure. However, each lattice constant shows complicated behavior. In particular, in the ac plane, the lattice constants a and c exhibit oppositeR_Z dependences, and the x dependence of the β angle gets weaker as R_Z increases. Overall, however, the area of the ac plane (ac sinβ) shows rather normal dependence on R_Z and x. All these results indicate that the cation effect on the unit cell is similar to the pressure effect but is not isotropic, which would be a feature of chemical pressure.

In molecular conductors, including the present system, the electronic structure in the vicinity of the Fermi level is well described by the tight-binding method based on the extended Hückel molecular orbital calculation. Intermolecular transfer integrals between frontier orbitals (in general, HOMOs in a donor system and LUMOs in an acceptor system) calculated from crystal data determine the energy bands around the Fermi level. Figure 4a shows a set of intermolecular transfer integrals for the β'-type Pd(dmit)₂ salts. (Note here that the C-centered unit cell is reduced to a primitive cell that contains only one dimer in the anion layer. Interlayer interactions are weak, and thus neglected in this work). Although the Pd(dmit)₂ molecule is an acceptor, the conduction band of the β'-type Pd(dmit)₂ salts originates from the HOMO. This is a quite unique feature of this system, and is due to the small HOMO-LUMO energy gap, ∆, and the strong dimerization [16]. In the dimer unit, each HOMO and LUMO in the monomer forms bonding and antibonding pairs with a dimerization gap. The dimerization gap is expressed as 2|T_A|, and both pairs have nearly the same gap value. In the β'-type Pd(dmit)₂ salts, because 2|T_A| is larger than ∆, the antibonding HOMO pair is located above the bonding LUMO pair and forms a half-filled conduction band. The LUMO-based energy bands are located near the HOMO-based conduction band and play an important role, which is another unique feature of the present system. Hereinafter, however, we focus on the intermolecular HOMO-HOMO interactions that govern the electronic structure near the Fermi level. In Table 2, the intermolecular HOMO-HOMO transfer integrals are listed for all the salts.

The strong dimerization allows us to take each dimer as an effective unit [2] and we can then redraw the unit cell with interdimer transfer integrals as shown in Figure 4b. In the dimer model, the conduction band is described by three interdimer transfer integrals, t_B, t_S, and t_r, as listed in Table 3. The interdimer transfer integrals relate to the intermolecular transfer integrals in Table 2 as t_B = T_B/2, t_S = (T_p + 2T_q)/2, and t_r = T_r/2. The largest transfer integral, t_B or t_S, is correlated with the band width W. For each salt, the triangular lattice formed by the interdimer interactions is approximately isosceles (t_B ≈ t_S), and thus the anisotropy of the triangular lattice can be estimated by the ratio t_r/t, where t = (t_B + t_S)/2. The anisotropy of the triangular lattice indicates the degree of geometrical frustration. As shown in Table 3 and Figure 5, t_r/t increases with increasing R_Z and x, and the Et₂Me₂Sb salt has an almost regular triangular lattice. The R_Z and x dependences of t and t_r are plotted in Figure 6. The increasing tendency of t_r/t is a consequence of decreasing t and increasing t_r. The decrease in t may be reasonable given that any increase in R_Z or x would expand the lattice and lengthen the intermolecular distances. On the other hand, the increase in t_r indicates some non-trivial structural change. In this system, the intermolecular transfer integrals are governed mainly by the overlap between the p orbitals of the sulfur atoms in the dmit ligand. Figure 7 shows the relation between T_r and the intermolecular S···S distances responsible for T_r. Notably, an increase in T_r results in shortening of d1 and d2, associated with the outer heteroring, while d3 and d4, associated with the inner heteroring, remain almost constant or slightly lengthen. These trends cannot be explained by changes in the lattice constants. When the dimer is viewed along the vector connecting the two Pd atoms, we notice that the Pd(dmit)₂ molecules in the dimer are not completely eclipsed (Figure 8). In particular, there is significant displacement of the outer parts, which indicates that the Pd(dmit)₂ molecule is slightly arched, with both ends of the molecule slightly deviating from the expected linear axis. Indeed, the arch-shaped distortion of the molecule can be detected in the bond lengths and angles, for example, in the Et₂Me₂Sb salt (Figure 9). In addition, Figure 8 suggests that the curvature depends on the counter cation. We tentatively describe the degree of arch-shaped distortion by defining the torsion angle of S–Pd–Pd–S (Φ) within the dimer (Figure 10). Figure 10 indicates that Φ changes systematically depending on the counter cation (R_Z and x), which is consistent with the above-mentioned trends for the intermolecular S···S distances.

The arch-shaped distortion affects not only T_r but also the other intermolecular transfer integrals. With increasing cation size (R_Z and x), the Pd(dmit)₂ molecule becomes distorted, as indicated by the green arrows in Figure 11. The distortion reduces the face-to-face overlap along the stacking direction that correlates with T_B (Figure 11). The distortion lengthens the intermolecular S···S contacts responsible for T_p, which is consistent with the observed relation between T_p and the intermolecular S···S distances responsible for T_p (Figure 12). In contrast to the results for T_r (Figure 7), a reduction in T_p correlates with elongation of all the S···S distances (Figure 12). As a result, the three interdimer transfer integrals, t_B, t_S, and t_r, are reasonably correlated with Ф (Figure 13).

In the Mott system based on dimer units, the intradimer transfer integral (T_A) is approximately correlated with the effective on-site Coulomb energy of the dimer [15]. Figure 14 shows the T_A values of β'-type Pd(dmit)₂ salts as a function of R_Z. Although the T_A values are larger than the interdimer transfer integrals, their variation is within 5%, which is smaller than the observed variations in t and t_r. As a result, we can say that the effect of the counter cation on T_A is relatively weak. The cation dependence of T_A appears complicated and does not show a systematic trend. In Figure 15, the intermolecular S···S distances responsible for T_A are plotted against T_A. (Note here that the T_A contribution of the Pd···Pd contact is small because the contribution of the d orbital of Pd to the HOMO is small). The inner S···S distances (d1 and d2) are shorter than the outer S···S distances (d3 and d4), which implies that the molecular plane is warped, as if the two Pd(dmit)₂ molecules in the dimer act to repel each other. For these two distinct groupings, the S···S distances are almost constant within each group over all the salts. The terminal S···S distances (d5) are largest and are slightly scattered without any clear tendency. The relation between T_A and Ф is shown in Figure 16, which suggests there are two distinct groups among the salts. The salts with Z = Sb show larger T_A values, while the other salts (Z = P, As) exhibit a chevron-shaped Ф dependence. Since it is expected that the dimer with Ф =0 would have well-eclipsed overlapping modes, this chevron-shaped dependence on Ф seems reasonable. On the other hand, the origin of the larger T_A values for the Sb series remains an open question.