Skip to Content
  • Review
  • Open Access

1 January 2003

Development of a competence scale for brief psychodynamic investigation: a pilot study 1

,
,
,
and
1
Centre for Psychoanalytic Psychotherapies Studies, University of Lausanne
2
Médecin responsable Hôpital psychiatrique du secteur Fondation de Nant CH-1804 Corsier-sur-Vevey
*
Author to whom correspondence should be addressed.
1
This research was supported by the Swiss National Fund for Scientific Research (FNRS), Grant No 3200-05901.

Summary

Background: The Brief Psychodynamic Investigation (BPI) is an ultra-brief psychodynamic psychotherapy used at the time of intake interviews in order to investigate a patient’s difficulties. Inspired by the brief psychodynamic psychotherapy tradition, BPI focuses mainly on the patient’s initial request for treatment, which is then explored for a maximum of four sessions. A therapist’s competence in using proper techniques, which refers to the level of skill shown by the therapist in delivering the treatment, is one among multiple variables that may influence the psychotherapeutic process in BPI. In order to assess this variable, we have developed an instrument: the brief psychodynamic Investigation Competence Scale (ICS), composed of 33 items divided into 5 subscales. The first two subscales refer to the therapist’s general and psychoanalytic attitude, whereas the other three subscales refer to his competence in investigating and interpreting, the BPI’s specific techniques and the therapist’s global competence. The aim of this study was to validate the ICS by testing its (a) inter-rater reliability, (b) internal consistency, (c) content validity and (d) construct validity. Method: A pilot study was done on 16 BPIs, half of which were done in consensus sessions. The interviews used in this study were taken from a larger project on the development of early alliance during BPIs (Lausanne Early Alliance Project, LEAR). All interviews had been video or audio recorded. The subjects were chosen amongst adults requesting an appointment with a therapist from the Lausanne University Adult Psychiatry Department’s outpatient clinic. The patients had been diagnosed as presenting anxiety, mood disorders or personality disorders. Seven therapists participated in this study: 4 therapists were considered to be experts in BPI while the other 3 were considered to be junior therapists.Two raters participated in assessing the 16 available BPIs according to a rating manual. The raters were blinded to the outcome of the therapeutic intervention. Results: Inter-rater reliability was assessed using intra-class correlations (ICC). ICCs ranged from 0.54 to 0.84 with an average of 0.71: these scores are in the range of scores obtained in other studies on adherence and competence scales. Significant scores are also obtained for the internal consistency of the scale. The content of the ICS was found significant by 4 independent expert therapists and the author of BPI. Finally, using the ICS permitted to discriminate between senior (>5 years of experience in BPI) and junior (<5 years experience) therapists. Conclusions: Initial results concerning the psychometric properties of the ICS are promising. The instrument reflects four basic aspects of psychodynamic investigation: (1) the general attitude in receiving the patient, (2) the psychoanalytic attitude during the investigation, (3) the exploration of conflicts and repetitions and, (4) the meaning given to the patient’s conflicts in an initial interpretation. The limits of this pilot study must be pointed out, regarding mainly: (1) the relatively small size of the sample, (2) the fact that the raters sometimes knew the therapists and how much clinical experience they had, and (3) a limitation related to the necessity to give clear explanations on how to use the instrument.

Introduction

With recent efforts in psychotherapy research to demonstrate the scientific value of therapeutic work, much attention has been given to the question of manualised treatments where both the theoretical and technical aspects of an intervention are systematically described [1,2,3,4,5]. These manuals often serve as a reference in training and supervising psychotherapists. Although some question the clinical use of these methods [6,7], researchers can only agree that they may serve as a basis for learning technical skills and could minimise the effect of other variables related to the therapist. Research projects could then be repeated and results compared without the problematic interference of these therapist-related variables. Hence, many instruments have been developed in order to determine whether a treatment is applied according to its most important techniques and principles. In general, these scales [6,8,9,10] are considered: (1) to be a means of assessing an intervention and determining whether it is done according to the procedures, (2) to allow a quantitative assessment of the effects of a given therapeutic intervention, and (3) to offer the opportunity to examine the effects of training on a therapist’s techniques.
Two types of scales have so far been suggested. The first set of scales assesses adherence and refers to the extent to which a therapist uses interventions and techniques prescribed by the treatment manual and avoids the use of intervention procedures proscribed by the manual. The second set of scales assesses competence which refers to the level of skill shown by the therapist in delivering the treatment [11]. Hence, adherence and competence scales cannot be considered to be completely independent. In our opinion most scales already used in research and clinical settings are constructed according to a continuum. At one end of this continuum we often find a definition of adherence where descriptive and quantitative elements predominate while, at the other end, is represented the idea of competence, mostly in a prescriptive and qualitative way.

Adherence and competence scales for psychodynamic psychotherapies

As can be seen in Table 1, most scales developed over the last 10 years share many similarities. Most of them assess either adherence [12,13,14] or competence [15,16]; or, in some rare cases, both concepts together [17,18]. These scales can often be distinguished in regards to the different dimensions assessed by their subscales. Furthermore, they were elaborated for different reasons and motives, be it in order to assess the influence of adherence and competence as well as of training of therapists on therapy outcome, to discriminate between different therapeutic interventions or to differentiate between specific and non-specific elements involved in a therapeutic intervention.As such, these scales reveal different levels of complexity and specificity of the construct under examination. As Waltz et al. [11] have noted, they then differ in the expertise needed to make proper use of the measure. Inevitably, this has a direct influence on the reliability of these instruments, occasionally reported as surprisingly low. Furthermore, most measures were devised using various sources of information, be it videotapes or transcripts for example, which may potentially influence measures of adherence and competence [19]. In this same line of thought it also appears that these instruments were elaborated on the basis of different manipulation checks such as random 15-minute segments or entire sessions. They finally concerned therapy sessions with different disorders, despite findings indicating that a deviation from the manualised therapy procedures may be indicated with difficult patients [17]. In general, these differences can be thought to have many consequences for the use and significance of such scales.
Table 1. Adherence-competence scales for psychodynamic psychotherapies.
Two common elements can nonetheless be found in many of these instruments. Most of them address the general attitude of the therapist, including such aspects as the assessment of specific and non-specific therapeutic techniques like a non-judgmental attitude, proper listening and the clarity of interventions. The second similarity in such instruments is that they assess the specific techniques which characterise and differentiate therapeutic models as described in manuals such as supportive and expressive techniques [18]. Studies using these instruments have demonstrated that they were capable of differentiating between different therapeutic techniques [14]. On the other hand, little–if any–is known as to whether these instruments could distinguish between different techniques within one psychotherapeutic method. Furthermore, training therapists in a given method has been shown to increase their competence and adherence to the method [17]. However, this increase cannot systematically be associated with better therapeutic outcome. Although some studies [15,18] have shown that competence scores and outcome are indeed related, others have given results that are somewhat uncertain [16].

The Brief Psychodynamic Investigation

The Brief Psychodynamic Investigation (BPI) [20] is an ultra-brief psychodynamic psychotherapy used at the time of intake interviews in order to investigate a patient’s difficulties. Inspired by the brief psychodynamic psychotherapy tradition, BPI focuses mainly on the patient’s initial request for treatment, which is then explored for a maximum of 4 sessions. The therapist’s goal during the initial interview is to elaborate a psychodynamic hypothesis leading to a first interpretation (referred to as the “initial interpretation”) about the patient’s conflicts in regard to his seeking treatment. This interpretation is then shared with the patient by also considering his personality structure. The second and third session is used to elaborate and develop the therapist’s initial hypothesis and to further the patient’s understanding of it. The fourth session is then used to conclude and summarise the investigation and to determine the specifics of further treatment. A fifth and last session is finally used to complete DSM-IV diagnosis using a semi-structured investigation. The patient can then decide whether he wishes to continue treatment and examine what type of therapy he could most benefit from.
In summary, the specific goals of BPI are: (1) to define the patient’s problems and motives regarding his request for therapy and to formulate an initial psychodynamic hypothesis according to his core conflictual relationship theme and personality structure; (2) to present the patient with the initial interpretation in order to clarify his motives for seeking therapy and to assess his resources and capacities to invest in a therapeutic process; (3) to develop a therapeutic alliance; and (4) to discuss with the patient the different options and possibly most effective interventions available.
This pilot study has two goals. First, it aims at presenting a competence scale for the BPI. Second, it presents initial reliability and validity data for this measure with a sample of 16 subjects.

Method

Sample
All the interviews used in this study were taken from a larger project on the development of early alliance during BPIs (Lausanne Early Alliance Project, LEAR). All interviews had been video or audio recorded. Furthermore, this data was completed with regular assessments using different instruments such as the Helping Alliance Questionnaire [21] ([22] for the French translation) after each therapy session. In order to test our competence scale for BPIs, a total of 16 subjects were used, 8 of them having been treated by experienced therapists while the remaining 8 had been treated by junior therapists. The cases were chosen by the therapists themselves on the basis of whether or not they could be considered as good examples of the application of the method.

Patients

All subjects were chosen amongst adults (over 18 years of age) requesting an appointment with a therapist from the Lausanne University Adult Psychiatry Department’s outpatient clinic. The patients had been diagnosed as presenting anxiety, mood disorders or personality disorders. Patients presenting psycho-organic complications, mental retardation, uncontrolled substance abuse, schizophrenia, bipolar disorders or antisocial personality disorder were excluded from the study. All gave informed consent to participate in the project.

Therapists

The seven therapists who participated in this study are affiliated to the Adult Psychiatry Department. Six of them are licensed psychiatrists and psychotherapists while the last therapist is a licensed psychologist and psychotherapist. Four therapists were considered to be experts in BPI (i.e. over 10 years of experience as BPI psychotherapists) while the other 3 were considered to be junior therapists with 5 to 10 years of experience in psychodynamic psychotherapy but only 2 to 5 years of experience in BPI.
The BPI Competence Scale
The BPI Competence Scale is composed of 33 items divided into 5 subscales. The first two subscales refer to the therapist’s general attitude, whereas the last 3 subscales refer to his competence in using the BPI’s specific techniques (Table 2):
Table 2. ICS subscales, representative items and scoring points.
  • The therapist’s general attitude (7 items). These items are generally considered to be nonspecific factors in the therapist’s attitude.
  • The therapist’s psychoanalytic attitude (7 items). These items refer to the parameters associated with the therapist’s psychoanalytic attitude (i.e., neutrality, interpreting resistance). In order to highlight and assess the specific impact of the BPI’s techniques according to a psychoanalytic model (subscales 3 and 4), emphasis was put on the attitude expected in a psychoanalytic psychotherapy.
  • The therapist’s competence in investigating (9 items). The items refer to the competence necessary for proper investigation (i.e. collecting relationship episodes, exploring core conflictual themes).
  • The therapist’s competence in interpreting (7 items). These items refer to the format, the content and the significance of an interpretation.
  • The therapist’s global competence (3 items). These items refer to a general assessment of the therapist’s competence in using adequate technique and creativity. The last item of this subscale concerns the patient’s apparent difficulties.
Each item is scored using a five point Likert scale. Each item and the scoring system are explained in detail in a scoring manual (cf. Table 2).

Elaborating the ICS

During the initial construction of the scale, each item was scored both in regard to adherence and competence. It then became obvious that the two methods were highly correlated. Furthermore, items referring to adherence appeared to be much more difficult to score as well as less consistent (Cronbach’s α) than was the case for the competence items. For these reasons, only one scale was thereafter used. Modifications brought to the ICS regarding content validity and following remarks from experts in BPI were done before the training of a second rater and further validation procedures were initiated.
Coding procedures
Raters: Two raters participated in assessing the 16 available BPIs. Although the raters were blinded as to the outcome of the therapeutic intervention, it must be said that in some cases the raters might have known the therapist.
Alliance: The therapeutic alliance was assessed using the Helping Alliance Questionnaire (HAq) [21], which was filled out by the subjects after each of the four BPI sessions.
Scoring procedures: As the most technical aspects of BPI are to be used during the initial interview, further analysis was done using material from the intake session only. The raters scored the interview independently using video recording and transcript (inter-rater reliability). Fifty percent of the ratings were done in consensus in order to insure greater validity.

Results

Reliability of the ICS
Two aspects of inter-rater reliability were tested for the BPI competence scale: (a) inter-rater reliability and (b) internal consistency of the global scale as well as of the different subscales.
Inter-rater reliability: Inter-rater reliability was assessed using intra-class correlations (ICC). ICCs ranged from 0.54 to 0.84 with an average of 0.71 (Table 3). These results are satisfactory when considering the relatively small size of the sample.
Table 3. Inter-rater reliability and internal consistency of the ICS scales (n = 16).
Internal consistency: Significant results were obtained for the four subscales as well as for the complete scale with Cronbach’s α ranging from 0.81 for the general attitude subscale to 0.89 for the competence in investigating subscale. Cronbach’s α for the 30 items of the global scale was 0.89.
Validity of the ICS
Two aspects of the ICS’s validity were examined: (a) content validity, that is the instrument’s capacity to grasp the subtleties of the BPI’s techniques and; (b) construct validity, that is the instrument’s capacity to discriminate a variable related to the therapist’s technique. More specifically, we hypothesised that the ICS should differentiate between the different therapists according to their clinical experience.
Content validity: Five psychotherapists were chosen on the basis of their experience in using BPI and questioned about the ICS’s validity. They said of most of the instrument’s items to be important and good descriptors of the BPI techniques. On the other hand, some criticism was made regarding (a) an occasional vague or ambiguous formulation of items relating to the psychoanalytic investigation or attitude, (b) the unnecessary repetition of certain items, and (c) the necessity to give precise rating instructions to the raters as well as clear examples for each technique described. These comments led to reformulating certain items and to the elimination of 5 redundant items (the initial form suggested a total of 38 items).
The correlations between the different subscales (Table 4) are in agreement with our clinical experience. For instance, there appears to be interdependency between competence in general attitude and competence in investigation as well as between competence in investigating and competence in interpreting.
Table 4. Correlations between subscales.
Construct validity: As suggested in our initial hypothesis, the differences between average scores of senior and junior therapists indicate significantly that the later have lesser success in BPI (Table 5). In general, alliance scores are not correlated with the ICS although the psychoanalytic attitude subscale is (r = 0.48, p = 0.05).
Table 5. Competence scores according to experience (Mann-Whitney).

Discussion

With inter-reliability (ICC) scores ranging from 0.54 to 0.84, results confirm that the ICS is a reliable instrument. Hence, even the techniques specific to BPI could be reliably assessed by independent raters. In general, these reliability scores are similar to those reported by Hollon et al. [23] in assessing interpersonal psychotherapy, cognitive-behavioural therapy and clinical management. They can also be favourably compared with results reported by O’Malley et al. [15], Svartberg [16] and others, e.g. Shapiro and Startup [13], although they are slightly inferior to those mentioned by Butler and Strupp [6]. Hence, this pilot study on a small sample of BPIs is therefore encouraging for future developments.
Many means could nonetheless be used in order to increase reliability scores. For instance, it may prove necessary to increase the number of judges in order to increase reliability coefficients to greater levels. It may also be useful to make use of–or better train raters as well-trained experts may reach greater agreement in assessing the different treatment procedures. Finally, greater details could be given concerning appropriately absent interventions [18] as refraining from making a specific intervention may reveal itself to be more appropriate sometimes and in some cases. Unlike most behavioural therapies, psychodynamic or psychoanalytically oriented therapies do not imply strict technical prescriptions or timedefined interventions but rather prescribe interventions based on a psychodynamic understanding of the fluctuations within a session. Hence, making use of a psychodynamic technique such as the interpretation of a given conflict may not seem warranted at all times. As such, raters must also be able to determine whether the absence of a given technique is fruitful to the therapeutic process and ultimately respectful of the treatment procedures. Furthermore, although competence often presupposes adherence, adherence does not necessarily imply competence [11]. For instance, a supportive and warm attitude may be useful at time X with a given patient while not being adequate with another. For this reason, our assessment of competence is both an indicator of adherence to the BPI’s specific techniques and of competence in adapting one’s behaviours to the patient and to the therapeutic settings. More research is nonetheless necessary in order to fully understand the implications of these processes.
Initial results regarding the instrument’s face and content validity are promising and can also be favourably compared with results on other similar scales. Its psychometric properties are mainly at the level of its subscales which assess competence in terms of the four fundamental aspects of BPI: (1) initial contact (subscale: general attitude); (2) adapting psychoanalytic techniques to the first contact with the patient (subscale: psychoanalytic attitude); (3) exploring the conflicts and repetitions (subscale: exploration of the psychodynamic hypothesis); and (4) making sense of the conflicts (subscale: initial interpretation). The acceptable levels of internal consistency–ranging from 0.81 to 0.89–reveal that the items may measure the same concepts. Furthermore, questioning of expert therapists and the author of BPI indicated that the items reflected the specific content and the general spirit of such a psychodynamic investigation.
In order to further examine the relationship between the different subscales, correlations were also done. It then appeared that the specific competence skills (competence in investigating and competence in interpreting) are correlated with the therapist’s general attitude, although it is often thought that they are opposite and function independently. The correlations between these subscales indicate that psychodynamic techniques such as dynamic investigation and understanding of a patient’s conflicts and the following action of interpreting cannot easily be separated from the therapist’s general attitude. It may even be argued that in initial sessions the therapist’s attitude reflects and is centred around his attempts to dynamically understand his patient and eventually formulate an initial interpretation of his conflicts [24]. On the other hand, as not all subscales are correlated, it does not appear that the ICS can only be the reflection of the therapist’s interviewing style or interpersonal manner. As such, the therapist’s psychoanalytic attitude cannot be said to be another expression of what Butler et al. [17] term the “good guy factor”. Finally, the absence of any significant correlation between the therapist’s psychoanalytic attitude and what are traditionally said to be psychoanalytic acts (i.e., investigating and interpreting) is somewhat puzzling. It may hence be argued that these subscales refer to two different themes involved in any such investigation. For instance, a psychoanalytically oriented therapy often involves that the therapist give more room to the patient and temporarily function on a more passive mode in order to share the patient’s intimacy, fantasies and representations [25]. As such, free association and temporary regressions are considered to be part of the process. Opposite to this can then be found the therapist’s more active stance, which involves direct investigation of specific themes and the formulation of an initial interpretation. These two different realms, a passive and an active one, are necessary conditions for any psychoanalytic therapy or investigation to take place. Furthermore, they apparently mobilise very different attitudes in the therapist.
Regarding the discriminant validity of the ICS, we found preliminary evidence that the instrument is helpful in differentiating therapists according to their clinical experience. This is true despite the fact that the junior therapists we assessed already had extensive clinical training (4 years of clinical and theoretical training in BPI).The importance of experience in using all BPI-related therapeutic techniques can hence be seen when examining BPI sessions. This is particularly true regarding the therapist’s general attitude and his competence in exploring the psychodynamic hypothesis. In agreement with our initial hypothesis, experienced therapists showed greater competence on all subscales.
A further aspect of process research we examined concerned the relations between ICS measures and therapeutic alliance. When comparing competence scores for the different subscales with alliance at the last therapy session, it is interesting to note that what is most correlated with alliance at that moment is the therapist’s ability to maintain a psychoanalytic attitude during the intake interview. This is most interesting when one considers that an intake interview usually implies more activity on behalf of the therapist than in later sessions. This could possibly be understood as a means to treat or to at least put a temporary stop to the repetition of core conflictual issues in the context of an initial contact. It can then be considered to be a specific aspect of psychoanalytic intervention that could possibly allow change and better alliance.
Finally, the limits of this pilot study must be pointed out, regarding mainly: (1) the relatively small size of the sample and (2) the fact that the raters sometimes knew the therapists and how much clinical experience they had. The first limitation implies that the small sample size for the therapists involved in the study leads to some redundant data. This difficulty as well as the limitation resulting from the fact that some judges knew the therapists and their level of experience arise from restrictions in the means available for most pilot studies and will be addressed in future work.
A third limitation is related to the necessity to give clear explanations on how to use the instrument. This difficulty seems obvious when considering the better inter-rater reliability scores for the BPI’s specific subscales (exploration of the psychodynamic hypothesis and initial interpretations). As in most therapy procedures, such specific techniques are easier to describe, hence leading to clear rating instructions. On the other hand, the definition and the rating instructions related to the two non-specific subscales (general characteristics of the therapist and psychoanalytical attitude) are much more difficult to define as they address more general and global attitudes of the therapist. As such, the rating of the therapist’s psychoanalytical attitude appears to be dependent on the rater’s familiarity with the psychoanalytic theories and techniques. This highlights the fact that extensive training of raters is necessary and that elaborating a manualised scoring procedure would in that sense be essential.
However, despite these limitations initial results are encouraging. The next step in addressing these questions would then be to increase sample size in order to: (a) assess the factor structure of the instrument, (b) emphasise the relation between competence and results from the investigation, and (c) examine the effects of training and clinical experience on the therapists’ competence. The ICS could then become a handy measure to accurately assess the performance of trainees and the effects of training.

References

  1. Beutler, LE; Machado, PP; Allstetter, S. Therapist variables. In Handbook of Psychotherapy and Behavior Change; Bergin, AE, Garfield, SL, Eds.; Wiley Press: New York, 1994; pp. 229–69. [Google Scholar]
  2. Strupp, HH; Hadley, SW. Specific vs non-specific factors in psychotherapy: a controlled study of outcome. Arch Gen Psychiatry 1979, 36, 1125–36. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  3. Crits-Christoph, P; Baranackie, K; Kurcias, JS; Beck, AT; Carroll, K; Perry, K; et al. Meta-analysis of therapist effects in psychotherapy outcome studies. Psychotherapy Research 1991, 1, 81–91. [Google Scholar] [CrossRef] [Scilit]
  4. Luborsky, L; Barber, JP. Benefits of adherence to psychotherapy manuals, and where to get them. In Psychodynamic Treatment Research: Handbook for Clinical Practice; Miller, NE, Luborsky, L, Barber, JP, Docherty, JP, Eds.; Basic Books: New York, 1993; pp. 211–25. [Google Scholar]
  5. Kernberg, OF; Clarkin, JF. Developing a disorder-specific manual: the treatment of borderline character disorder. In Psychodynamic Treatment Research: Handbook for Clinical Practice; Miller, NE, Luborsky, L, Barber, JP, Docherty, JP, Eds.; Basic Books: New York, 1993; pp. 227–44. [Google Scholar]
  6. Butler, SF; Strupp, HH. Effects of training experienced dynamic therapists to use a psychotherapy manual. In Psychodynamic Treatment Research: Handbook for Clinical Practice; Miller, NE, Luborsky, L, Barber, JP, Docherty, JP, Eds.; Basic Books: New York, 1993; pp. 191–209. [Google Scholar]
  7. Addis, ME; Krasnow, AD. A national survey of practicing psychologists’ attitudes toward psychotherapy treatment manuals. J Consult Clin Psychol 2000, 68, 331–9. [Google Scholar] [CrossRef] [PubMed]
  8. Klerman, GL; Rounsaville, B; Chevron, E; Neu, C; Weissmann, MM. Interpersonal Psychotherapy of Depression; Basic Books: New York, 1988. [Google Scholar]
  9. Luborsky, L. Principles of Psychoanalytic Psychotherapy: A Manual for Supportive Expressive Treatment; Basic Books: New York, 1984. [Google Scholar]
  10. Rounsaville, B; O’Malley, S; Foley, S; Weissmann, MM. The role of manual guided training in the conduct and efficiency of interpersonal psychotherapy for depression. J Consult Clin Psychol 1988, 56, 681–8. [Google Scholar] [CrossRef] [PubMed]
  11. Waltz, J; Addis, ME; Koerner, K; Jacobsen, NS. Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. J Consult Clin Psychol 1993, 60, 620–30. [Google Scholar] [CrossRef] [PubMed]
  12. Hill, CE; O’Grady, KE; Elkin, I. Applying the collaborative study rating scale to rate therapist adherence in cognitivebehavior therapy, interpersonal therapy, and clinical management. J Consult Clin Psychol 1992, 59, 73–9. [Google Scholar] [CrossRef] [PubMed]
  13. Shapiro, DA; Startup, M. Measuring therapist adherence in exploratory psychotherapy. Psychotherapy Research 1992, 3, 193–203. [Google Scholar] [CrossRef] [Scilit]
  14. Startup, M; Shapiro, DA. Therapist treatment fidelity in prescriptive vs. exploratory psychotherapy. Br J Clin Psychol 1993, 32, 443–56. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  15. O’Malley, SS; Foley, S; Rounsaville, BJ; Watkins, JT; Sotsky, SM; Imber, SD; et al. Therapist competence and patient outcome in interpersonal psychotherapy of depression. J Consult Clin Psychol 1988, 56, 496–501. [Google Scholar] [CrossRef] [PubMed]
  16. Svartberg, M. Manualization and competence monitoring of short-term anxiety-provoking psychotherapy. Psychotherapy 1989, 26, 564–71. [Google Scholar] [CrossRef] [Scilit]
  17. Butler, SF; Henry, WP; Strupp, HH. Measuring adherence in time-limited dynamic psychotherapy. Psychotherapy 1995, 32, 629–38. [Google Scholar] [CrossRef] [Scilit]
  18. Barber, JP; Crits-Christoph, P. Development of a therapist adherence/competence rating scale for supportiveexpressive dynamic psychotherapy: a preliminary report. Psychotherapy Research 1996, 6, 81–94. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  19. Chevron, ES; Rounsaville, BJ. Evaluating the clinical skills of psychotherapists: a comparison of techniques. Arch Gen Psychiatry 1983, 40, 1129–32. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  20. Gilliéron, E. Manuel de psychothérapies brèves; Dunod: Paris, 1997. [Google Scholar]
  21. Alexander, LB; Luborsky, L. The Penn helping alliance scales. In The Psychotherapeutic Process: A Research Handbook; Greenberg, LS, Pinsof, WM, Eds.; Guilford Press: New York, 1986; pp. 325–66. [Google Scholar]
  22. Gerin, P; Dazord, A; Sali, A. Psychothérapies et changements. Méthodologie de leur évaluation. [Psychotherapies and changes. Methodology of their evaluation]; Presses Universitaires de France: Paris, 1991. [Google Scholar]
  23. Hollon, SD; Evans, MD; Elkin, I; Lowery, HA. System for rating therapies for depression; Paper presented at the Annual Meeting of the American Psychiatric Association: Los Angeles, CA, 1984. [Google Scholar]
  24. Despland, JN; de Roten, Y; Despars, J; Stigler, M; Perry, JC. Contribution of patient defense mechanisms and therapist interventions to the development of early therapeutic alliance in a brief psychodynamic investigation. J Psychother Pract Res 2001, 10, 155–64. [Google Scholar] [PubMed]
  25. Angelergues, J. Contre-transfert et création. Revue Française de Psychanalyse 1994, 5, 1535–8. [Google Scholar] [CrossRef] [Scilit]

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.