Freelancing Models for Fostering Innovation and Problem Solving in Software Startups: An Empirical Comparative Study

Abstract

Context: freelancers and startups could provide each other with promising opportunities that lead to mutual growth, by improving software development metrics, such as cost, time, and quality. Niche skills processed by freelancers could help startups reduce uncertainties associated with developments and markets, with the ability to quickly address market issues (and with higher quality). This requires the associations between freelancers and startup to be long-term, based on trust, and promising agreements driven by motivations (leading to the growth of both parties). Freelancers could help startups foster innovations and undertake software development tasks in better ways than conducted in-house, if they are selected using informed decision-making. Objectives: the paper has three objectives, (1) to explore the strategies of startups to outsource software development tasks to freelancers (termed as freelancing association strategies); (2) to identify challenges in such outsourcings; and (3) to identify the impacts of outsourcing tasks to freelancers on overall project metrics. The overall objective is to understand the strategies for involving freelancers in the software development process, throughout the startup lifecycle, and the associated challenges and the impacts that help to foster innovation (to maintain competitive advantages). Method: this paper performs empirical studies through case studies of three software startups located in Italy, France, and India, followed by a survey of 54 freelancers. The results are analyzed and compared in the identification of association models, issues, challenges, and reported results arising because of such associations. The case study results are validated using members checking with the research participants, which shows a higher level of result agreements. Results: the results indicate that the freelancer association strategy is task based, panel based, or a hybrid. The associations are constrained by issues such as deciding pricing, setting deadlines, difficulty in getting good freelancers, quality issues with software artefacts, and efforts to access freelancer work submissions for reward. The associations have a positive impact on software development if there is availability of good freelancers (which lasts long for various tasks). The paper finally provides a freelancing model framework and recommends activities that could result in making the situation beneficial to both parties, and streamline such associations. Fostering innovation in startups is, thus, a trade-off situation, which is limited and supported by many conflicting parameters.

1. Introduction

Freelancers are self-employed persons who have short term associations with particular employers. They are not bound under the obligations of standard working conditions (such as hours and physical presence), and they are not expected to serve a company for an extended period of time. The terms and conditions of the association are mutually agreed upon, including price, deadlines, milestones, work related quality standards, acceptance criteria, etc. Such freelancers have multiple ways of getting paid tasks from companies, which include personal links with companies, through referrals, and through freelancing platforms (middle agencies), such as freelancer.com (www.freelancer.com), Upwork (www.upwork.com), Guru (www.guru.com), etc. A freelancing platforms acts as an intermediary between the company (requester) and freelancer (requestee). The platform allows the requestee to upload the description of tasks to be outsourced to freelancers, and for freelancers to apply for them. The advantage of associating with the company through referrals and/or personal links is a price element of the freelancing offer, i.e., the price agreed upon between the requestee and freelancer does not include the fee, as charged by some freelancing platforms, as commission. However, referrals and personal links should be strong enough to overcome the trust factor that connects companies with freelancers. Freelancers must trust that companies will pay them after completion of the task. Similarly, companies must trust that freelancers will complete high-quality work within the deadline.

A startup is a temporary organization that offers innovative products to the market, and searches for repeatable and scalable business models by gathering validated learning, driven by continuous experimentations in the market. Continuous experimentations include the build–measure–learn cycle, to continuously test the assumptions (or hypotheses) about business model canvases (such as value proposition, customer segments, etc.), and modify them as per validated learning. This process continues until the startups find a business model that is scalable and repeatable, which leads to higher sales and growth of the firm. Once such model is identified; startups have to work in highly dynamic environments, full of evolving customer needs, technological innovations, competitor pressure, government regulation changes, etc. Thus, startups must incorporate innovative culture in their organizations to continuously invent creative ideas and implement them, as it generates value to the firms and the customers. Continuous innovation is key to maintaining a competitive advantage, to maintain higher customer-perceived values of the product offerings throughout the product life cycle. Startups turn into big companies once they capture sufficient market share, but they need to be innovative to avoid getting disrupted by competitors. Innovations could include basic research (as well as sustaining, breakthrough, or destructive [1]), and should be executed through the life cycle of the business in accordance with the trends in the environment.

Authors in [2], through a systematic mapping study, identified that software engineering research, in the context of startups, include concepts such as innovation/innovative, lack of resources, uncertainty, time-pressure, small team, highly reactive, and rapidly evolving. Startups are continuously performing experiments and innovate business models to grow in the market. They need to be highly reactive and agile to react to market changes and incorporate validated learnings. They have limited resources, including financial resources, human resources, intellectual, and physical resources. Limited resources must be employed optimally to reduce costs associated with delivering the value proposition with potential for growth. Startups have small teams and each team member usually has multiple roles to play, such as analyst, designer, coder, etc. Teams may not have the competencies (or niche skills) to handle unique challenges as identified during software development. Resource constraints increase startup dependencies on third parties, aligning them towards collaborative workings in the form of hiring freelancers and outsourcing software development tasks to third party agencies (such as software development firms).

The invention of creative ideas, their implementation, and adoption in the market leads to business growth, but is the litmus test for startups, as innovation requires optimal utilization of organizational resources. For instance, consider the two scenarios:

• The invention of diverse creative ideas requires the active support of customers, employees, channel partners, etc. The more ideas, the better the diversity and, hence, the better the options. This requires resource constrained startups to manage a flood of ideas from both inside and outside the organization by collecting ideas, evaluating ideas, selecting the promising ideas, implementing them, and finally market release. Idea management requires structured processes and infrastructure, which is limited in the startup context due to the lack of resources. This limits the ability of startups to employ open innovation with the external world on a larger scale.
• Implementing innovative ideas require software development skills. For instance, incorporating blockchain smart contracts require startups to have programmers with expertise in blockchain programming, such as SmartPy. Startups may find it difficult and costly to hire a blockchain expert, for such innovation, in local markets.

Startups could also find it difficult to identify a good software engineer, at low cost, in local markets, for undertaking software engineering activities, such as designing, coding, testing, etc. Software engineering tasks are complex problem solving tasks that require niche skills, which are harder and costlier to get, either as full time employee options or outsourced to a third party agency. Employing freelancers to handle challenging software development related tasks and innovation management could help startups to achieve the following:

• Bridge the skill-gaps startups have, at lower market costs (complex problem solving);
• Receive expertise from freelancers, in the form of innovative ideas and implementations (innovation management).

In general, freelancers help startups foster innovations by providing creative ideas derived by their expertise and/or implementation of ideas (complex software engineering problem). Freelancers, through their expertise and skills, help make business processes agile as they give them the ability to quickly respond to market changes, as reported in [3]. Big software companies have resources to support innovation and to find solutions to any complex software engineering problem in-house, without searching for part-time employer options, yet freelancers are being employed by many big software companies. The association of freelancers with startups will be very challenging when compared to big companies, due to the diverse challenges specific to startups [4]. Such associations also have diverse challenges (apart from generic association challenges), specific to software engineering activities that software startups decide to outsource [5]. This needs research work to be conducted, which could provide the necessary knowledge to startup founders (and their project team members) about the real time practices of software startups, involving freelancers for software development activities. The knowledge is applicable to any startup that decides to involve freelancers, for undertaking challenging tasks for which they do not have the resources.

Unfortunately, despite the benefits that freelancers could provide to startups, little empirical evidences are available in the literature about freelancer involvement in the different software engineering activities of software startups [6]. In other words, the literature lacks studies that highlight the way startups associate with freelancers, and the benefits this type of association brings to startups. The issues and challenges of such associations must be identified from two viewpoints, i.e., startups (requester) and freelancers (requestee). Thus, this exploratory study is best suited due to the limited studies available. The authors conducted the exploratory case study to explore software startup practices focused on freelancer involvement, by studying the cases in their real context.

The knowledge built by the case study will help the software startup community overcome their resource weaknesses by utilizing the strengths of freelancers, and convert these opportunities into long-term relationships. There are differences in terms of a long-term relationship with an employee of a company, and a long-term relation with a freelancers—with the freelance, there are no obligations placed on either party to expect or provide services as when required. The idea is to make such virtual associations more like employee relations, but with no legal obligations, which are more driven by motivation to contribute, by virtue of the niche skills a freelancer processes, and the growth opportunities provided by the requester. Authors in [7] suggested that the long-term association of freelancers with companies, handling multiple projects, helps to attain competitive advantages. Long-term associations help freelancers develop firm-specific human capital, bringing competitive advantages to the company.

This paper is structured as follows: Section 2 presents the survey of literature, highlighting case study guidelines, followed by the paper, and concepts related to the association of freelancers with startups. Section 3 presents the design of the conducted case study, including the presentation of the studied cases. The results of qualitative analysis applied on collected data are reported in Section 4. The freelancing model framework is presented in Section 5, followed by the presentation of the implications of the results in Section 6. The outcome of the case study is validated through member checking in Section 7. The limitations of the case study are reported in Section 8, followed by the conclusion and future work of the research (Section 9).

2. Literature Survey

2.1. Freelancers in Software Startups

Authors in [6] conducted a systematic mapping study to identify the research trends in freelancer supported software development. The research trends were reported against four parameters, which include (a) software development activities involving freelancers; (b) issues and challenges focused on by researchers; (c) type of research conducted; (d) temporal (time related) trends of research involving freelancers in software development. Research studies forming the basis of the systematic mapping study involved freelancers in software development in general, and, hence, they were not only focused on software startups, but the results were equally applicable for the startups as well. The results indicate that (a) research focus is on generic software development rather than on individual life cycle activities; (b) the number of empirical studies is limited; (c) number of studies proposing solutions and evaluating (using real dataset) in industrial settings are missing from the literature; (d) collaboration and coordination, developer recommendation, team formulation, and task decomposition are mostly focused challenges and issues, which are still focused on by the researchers.

The limited studies conducted in freelancer supported software development are equally applicable for software startups. However, software startups have challenges that are unique to them. This includes little or no operating history, limited resources, multiple influence from stakeholders, dynamic technologies, and markets [8]. Moreover, they have a less experienced team, high degrees of uncertainties, tight market release deadlines, are not self-sustained, highly innovative, and are highly reactive to changes and rapidly scaling requirements [9]. These challenges further limit the involvement of freelancers in software development with startups.

Authors in [2], through a systematic mapping study, identified that software engineering research, in the context of the startups, include concepts, such as innovation/innovative, lack of resources, uncertainty, time-pressure, small teams, and being highly reactive and rapidly evolving. These characteristics that pertain to startups (compared to the software companies) will affect the way startups collaborate with freelancers. For example, due to limited resources, the decisions to hire a freelancer (for which work, and under what conditions) would be based on educated guesses only. However, if such collaboration is done successfully, it is predicted that the startups will be able to discover game-changing business models (high value at low costs). To further strengthen these predictions, exploratory research is required to explore freelancer involvement in software development in startups. The literature lacks the studies that focus on exploring startup practices of involving freelancers in their software development activities and the challenges they face in doing so.

2.2. Freelancers in Generic Context of Software Development

The literature provides limited research studies that report on the diverse aspects related to the freelancing involvement in outsourced software development tasks. The software firms with limited budgets could undertake global projects by including freelancers in their global software teams in a cost effective and time efficient manner [10]. The outsourcing of tasks requires the accurate identification of freelancers amongst a pool of interested freelancers, which require careful consideration of their knowledge, skills reputation, expertise, reliability, and contacts to succeed in self-employment [10,11]. The literature provides the research studies that deal with analysis of the freelancing platforms [12], freelancer recommendations [13,14,15,16,17], task recommendation [18], freelancing platform efficiency (for instance, enhancing the capacity utilization of the freelancer platforms with guaranteed performance by reducing backlogs) [19], reporting issues, such as privacy, confidentiality, and so on [20], and freelancer recognitions [21].

The available literature provides studies, which are conducted across diverse freelancer involvements, but are limited in their scope and quantity. Limited studies: this limits the empirical support to the startups for involving freelancers in software engineering activities to foster continuous innovations, for gaining competitive advantages. Freelancer involvement in software startup is, thus, an unexplored research topic, and the majority of startups do not publicly announce their dependency on freelancers (this may be due to funding issues), proving to be a litmus test for exploring this complex research domain.

The exploratory study of a freelancer’s involvement in the startup context for software development activities is best suited through a case study research method. This will help to build a theory that will attract further research to provide an effective solution to startups, to optimally involve freelancers, and overcome the unique challenges they face. The overall objective is to study the freelancer involvement in the software startups from three perspectives, i.e., association strategies, challenges, and impacts, rather, providing the finer details about freelancer involvement across individual software engineering activities.

3. Research Design

3.1. Research Methodology

The paper meets the research objectives by finding answers to the formulated research questions, by executing the following research activities.

(a)

Case study with three software startup founders and senior software engineers. In other words, the newly joined or recently associated employees are not considered for interviews. The three cases were compared to identify meaningful propositions (cross-case analysis).

(b)

Case study with 54 freelancers, to collect their experiences about their associations with startups, the issues they face with them (if any), and the perceived value they bring to software projects. The 54 cases were compared to identify meaningful propositions (cross-case analysis).

(c)

Comparative analysis of findings in (a) and (b) to identify answers to stated research questions (cross-case analysis).

3.2. Case Study Guidelines

Authors in [22] presented the guidelines to conduct and report the case studies in software engineering. The case study research includes five steps i.e., case study design, data collection procedures, collecting evidence, analysis of collected data, and reporting. Case study design results in the identification of the case study protocol. The protocol contains the various case study elements as planned for the case study, such as case selection procedures, data collection procedures, analysis procedures, validity, ethical considerations, etc. Data collection methods are executed to collect data from various data sources, which then are analyzed using various analysis techniques leading to generation and/or testing of the hypothesis. The data are usually qualitative, but small quantitative data could also be collected, which are subjected to descriptive statistical analyses. The results of the case study are then reported to its intended audience using suitable reporting formats. The case study is executed considering the validity and ethical issues. The research is conducted to meet the formulated research objectives in the most trustworthy manner, satisfying the ethical issues involved in the research. Various checklists associated with the individual case study steps are provided to assist the researcher to conduct the case study in an effective and efficient manner.

3.3. Case Study Protocol

The research paper employs an exploratory case study to explore the way the software startups include freelancers for software development activities. The exploratory nature of the case study is necessitated because the literature lacks the manner concerning the development of the association between freelancers and companies, the implications of such associations, and the opportunities to improve their product offerings, overcoming resource limitations.

The case study follows the guidelines as proposed in [22] and executes the case study protocol, which is formulated as per the template proposed in [23], and given below:

3.3.1. Background

The paper aims to achieve the following objectives (Obj):

(a)

To explore the strategies of outsourcing software development tasks to the freelancers (Obj1).

(b)

To identify challenges faced by startups to hire freelancers for tasks to be outsourced (Obj2).

(c)

To identify the impact of outsourcing tasks to freelancers on overall project metrics (Obj3).

The paper tries to find answers to the following research questions (RQ):

RQ. 1

What strategies do the software startups execute for establishing the associations with freelancers?

RQ. 2

What are the issues and challenges faced in establishing and managing the associations?

RQ. 3

What impact the associations have on the software development metrics of software startups?

The research questions were formulated with the Population, Intervention, Comparison, and Outcomes (PICO) framework, as suggested in [24].

• Population: software startups.
• Intervention: involvement of freelancers.
• Comparison: not considered to better explore the problem domain.
• Outcomes: impact on software development metrics.

The relation between the research objectives and the research questions is given in Figure 1.

3.3.2. Design

Propositions/Hypothesis

The case study is exploratory in nature and, due to limited research available in the literature, the case study aims to build a theory related to freelancer supported software engineering in startups. Thus, it is not possible to formulate any propositions/hypotheses during the case study planning process.

The aim of this research is to build propositions/hypotheses, which are supported by a chain of evidences.

The case study design is the embedded multiple case study. This means that there are multiple cases that are studied by analyzing multiple units of analysis within each case. This research has the following design elements:

• Context: software development environment at startups.
• Cases: software startups and the freelancers (acting as requestor and requestee) for the outsourced tasks.
• Negative cases: freelancers who do not want to work with startups (but who had previously worked with them).
• Units of analysis: freelancer association strategies.

3.3.3. Selection

The startup cases are selected based on their ability to fulfil the purpose of the research study. The main criteria for their selection was that software startups should have been, at least, in the growth phase, and must have involved freelancers throughout their life cycle. This ensures the study is based on long-term freelancer associations, established by startups rather than studying the shorter-term associations, which could be episodic. Thus, the cases were selected using purposive sampling technique.

Freelancer cases were selected from three sources, i.e., those associated with the studied startups, those in the professional network of the authors, and those actively participating in online freelancing platforms with their contact information public. These freelancers were subjected to further screening based on their long-term experience (greater than 2 years) with the startups only (i.e., those working for big companies only were screened out).

3.3.4. Procedures and Roles

All of the researchers were involved in all procedures related to the design and execution of the case study protocol. Participation by all of the researchers helped ensure reliability of the study.

3.3.5. Data Collection

Data collection from the startup cases were done using interviews and observations. Analysis of archival records was quite impossible as the studied startups (cases) do not maintain documentation records.

Semi-structured interviews were conducted using a flexible interview guide. The interview guide includes background questions and specific questions. A semi-structured interview allowed the discussion to be initiated as per the interview guide (to ensure all important topics were covered), but proceeded in an elaborated fashion, as per the information brought by participants. The questions were open questions.

Observation was conducted at the working sites of the three startups. The outsourcing decisions were taken in all three startups in the meetings involving all software engineers and the founders. The objective of involving all employees was to keep everyone synchronized concerning decisions, and to invite different perspectives. Of course, small team sizes made such meetings feasible. Attending such meetings and observing the startup team interactions with the freelancers for a variety of outsourced assignments, helped researchers gather diverse perspectives about their associations with the freelancers.

The Indian startup was visited in the third week of October 2019, the France startup in the last week of October 2019 and first week of November 2019, and the Italy startup in fourth week of November 2019.

Startup founders, startups senior employees, and freelancers were the data sources. Collecting the data from multiple sources helped researchers attain data triangulation to ensure research validity and provide propositions/hypothesis along with the chain of evidences.

Data collection with the freelancers were conducted in two levels—questionnaire and group interviews. Initially, freelancers were approached with the questionnaire (after ensuring ethical implications and gathering their consent for participation). The objective of such a questionnaire is to streamline the further level of online interviews based on the freelancer’s initial responses and to draft the interview guide accordingly. This is because the data reported by the freelancers were analyzed and categorized into groups based on similar responses. The groups with similar reporting were subjected to semi-structured interviews as a group rather than individually. The online interviews were conducted using tools such as skype, WhatsApp, and voice calls to elaborate their responses and to clarify any misunderstandings. The interview sessions were guided by the flexible interview guide. Further, the perspectives brought by undertaking the observations at the startups (as mentioned above) helped further elaborate the interview session findings with the freelancers.

Ethical implications were fully considered by the researchers. The data instruments were made aware of the following (as suggested in [22]):

• Informed consent.
• Review board approval.
• Confidentiality.
• Handling of sensitive results.
• Inducements.
• Feedback.

The participants were informed about the study, its objectives, voluntary participation nature, the way confidential information will be handled, and expectations from participations. Informed consent was taken. Moreover, they were told that, to validate the results of the study, the results would be shared with them, and their feedback would be incorporated in the final results.

The field notes were prepared during interview sessions and were elaborated on after the sessions. The interview sessions were voice recorded, which helped to elaborate field notes and prepare the transcripts of the conversations. The field notes were discussed between two researchers, who jointly participated in interview sessions until final agreements were reached. A second round of interviews were conducted to verify the interpretations of the previous interview conversations made by researchers and to seek more information (that researchers felt would be interesting to ask).

3.3.6. Analysis

The collected data were qualitative in nature. The data were analyzed using the grounded theory approach. The transcripts were coded, and the codes merged to generate the series of propositions/hypotheses along with the evidence supporting them. The propositions were also compared with the findings brought by the analysis of qualitative data of other cases (cross-case analysis). The evidence that conflicted with the propositions helped researchers to modify them. The process of generation of the hypotheses and their testing were conducted in an iterative fashion, as mentioned in [25].

The researchers initially planned to provide descriptive statistics, as expected as an outcome for research question number 3, but the startups could only mention it in a qualitative way. One reason is that it is hard to quantify the benefits the freelancers brought on overall development metrics, such as cost, development time, and quality.

3.3.7. Plan Validity

The research focused on treating the validity issues with care. The objective was to enhance trustworthiness of the case study. The four threats to validity as proposed in [22] were handled in the following manner:

• Construct validity: multiple data sources were used, and study protocol was discussed between researchers and evaluated by the experts. The transcripts were shown to participants to ensure the common understanding of information between researchers and participants.
• Internal validity: as the research is exploratory, internal validity was not a threat to the study.
• External validity: the study results apply to software startups, but they equally hold for any software company. The reason is that non-startups are better placed than startups, in terms of their resource capabilities and, hence, they could better use freelancing opportunities. To ensure this, the founders of three startups (who worked at senior positions in software companies) were asked to share their feedback about the applicability of research results for the big companies. Their feedback supported the applicability. The feedback helped ensure external validity.
• Reliability: the use of data triangulation (use of multiple data sources and instruments), investigator triangulation (use of multiple researchers during research), theory triangulation (use of multiple perspectives to build theory, i.e., founders, software engineers, freelancers who preferred to work with startups, freelancers who never preferred to work with startups), and methodological triangulation (comparative data analysis of qualitative data collected from multiple sources and multiple instruments). Sharing study results with participants and getting their feedback ensured study reliability.

3.3.8. Study Limitations

The study could be extended by including the insights of the software startups that turned into big companies and involved freelancers in their activities. Moreover, they will be in position to provide quantitative data, but quantifying everything is difficult (with subjective judgement).

3.3.9. Reporting

The case study is reported for the researchers working in software engineering areas in the context of software startups. The report is also valuable for freelancers, startup founders, and their software engineers.

3.4. Details of Studied Units

Software startups were studied using interviews and observations. Three studied software startups were based in three countries, i.e., one was based in Italy, another in India, and the third based in France. These startups were founded by different founders (and co-founders), and these founders had experience establishing other successful startups. The startup life cycle is divided into three phases, i.e., startup, stabilization, and growth [26]. All startups were at the mid-point of the growth phase of the startup life cycle. None of the studied startups had turned into mature companies. The characteristics of the startups are given in

Table 1. Start-up categories.

S. No.	Start-Up Name	Country	Number of Employees	Number of Employees Interviewed (Including Founder)	Software Market Category
1.	A	Italy	8	3	Social sector
2.	B	India	7	2	Financial markets
3.	C	France	12	2	Educational software

.

The 54 freelancers were contacted through a questionnaire and online interviews, to collect their experiences and perceptions about the associations with the software startups. By including freelancers as cases in the case study, this helped researchers incorporate perspectives different from the founder and software engineer, i.e., modify or support propositions by including evidence given by freelancers. These freelancers were globally dispersed and not confined to the same geographical location.

To further enhance the validity of the case study, negative cases were also considered. Negative cases are the freelancers who never preferred to work with startups. Thus, the case study has two groups. One group (termed as F1) had 21 cases who worked with software startups (and perhaps with big companies); another group (termed as F2) had 33 cases who did not prefer to work with startups (but had, at any time in the past, worked with startups). All 54 cases are freelancers. The objective of including negative cases is to further strengthen the theory as built by the case study results, by including different perspectives.

The individual experience of freelancers in undertaking outsourced tasks will provide informative perspectives on how the constraints imposed by startups impacts the establishing and maintaining of the associations.

The characteristics of the freelancers, as employed, as cases in the research, are given in

Table 2. Freelancer categories.

S. No.	Category (Number of Freelancers)	Qualification	Experience	Skills
1.	F1 (21)	Students, researchers, few past employees.	Usually academic projects. For past employees, with live industrial projects.	Ranging from diverse to niche skills with less expertise.
2.	F2 (34)	Past Employees.	Experience of live projects with small, medium and large companies.	Niche skills with high level of expertise.

.

4. Result Analysis

4.1. Case Study Results

The study of three software startups through the series of interviews and observations is highlighted for each research question.

Table 3. Freelancer Strategies.

S. No.	Start-up Name	Strategy	Subcategory	Pricing
1.	A	Panel Based.	Crowdsourced panel based. Non- crowdsourced panel based.	For crowdsourced, the price mentioned in the open call for tasks to be crowdsourced is reached on basis of perception of startup team (if they have prior experience) or through the average of quotations from local markets. For non-crowdsourced panel based strategy, the prices are established by negotiating against market rates as benchmarks.
2.	B	Task Based.	Crowdsourced task based. Non-crowdsourced task based.	For crowdsourced, the price is established using either of three strategies: (a) Market rates. (b) Analysis of prices of similar tasks on various freelancing platforms. (c) Not mentioning the price and asking interested freelancers to mention it in their bids. For non-crowdsourced panel based strategy, the prices are established by negotiating against market rates as benchmarks.
3.	C	Hybrid. (Crowdsourced)	Conditional Panel based. Task Based.	Conditions about pricings is established considering the long-term vision, financial resources and future requirements of the freelancers. For task based, it’s similar to pricing models of startup A and B.

presents the final outcome of qualitative analysis.

RQ. 1

What strategies do the software startups execute for establishing the associations with the freelancers?

The strategies of the studied startups could be briefly described in Table 3. The data in Table 3 are a result of the analysis of qualitative data by using the grounded theory approach, resulting in the categories of freelancer association strategies and subcategories. The process of data analysis involved the iterative process of formulation and testing of propositions, resulting in meeting the objectives of the research. Cross-case analysis was employed, to take into consideration the multiple perspectives of study cases, and provide answers to formulated research questions.

Each startup has a different strategy to associate with the freelancer. Even the underlying principles of the models (crowdsourcing or non-crowdsourcing) are the same, yet they have different pricing mechanisms. The pricing depends heavily if they are dealing with a panel of selected freelancers, a crowd of freelancers, or individual freelancers. Yet, the price limits available in panel, task, and hybrid vary because of varying levels of freelancer associations, motivations, and startup team control. For example, in panel based pricing, the prices most likely reflect the pricing limits as set by startup strategies and better negotiations.

The researchers call the strategy employed by startup A as panel based freelancing because, under this freelancing, the startup hires the panel of freelancers. To establish long-term associations with freelancers, they are kept motivated as virtual employees through opportunities to participate in training, appreciation letters for high quality work, participation in official social events, meeting invitations, etc.

Further, various online courses they could enroll in are recommended, and they are given the opportunity to attend events organized in different venues, such as industrial seminars, educational training in partner universities, as its employees. They try their best to have a bigger size of the panel, and to maintain a demand–supply curve favorable to them to control prices (freelancing task price).

Whenever the startup requires freelancer services, they are given the opportunity to express their willingness to undertake freelancing tasks by quoting their bids, mentioning their perceived price for task (rate quotation), deadline, milestones with timelines, and work-related quality standards. Based on the collected bids, the company selects the most promising freelancer with a promising bid or may select them on the basis of competition, or through an interview.

The selection strategy depends on bids, efforts required, and urgency of the task to be performed. Thus, it may take the following forms:

• Crowdsourced panel based strategy (with competition among panel freelancers by inviting bids that could be simple bids (simple request to apply against call, termed as individual bids against open call), complete solutions to the outsourced tasks or abstract solutions (partial solutions submitted by freelancer, which represents their understanding about solutions to the outsourced tasks). The open call is made among the panel members.
• Non-crowdsourced panel based strategy (no competition, individual pick from the panel of freelancers, i.e., individual pick without open call). Individual pick means that no open call for task outsourcing is made among the panel and the most suitable freelancer is straightway selected for the execution of the task to be outsourced. The selection is on the basis of familiarity with work of the freelancer or ease of access to him. The individual freelancers are selected from the startup panel only, rather than selected from outside.

The opportunities are given to external freelancers to join the panel maintained by the startup. The variation (of the selection) of the different freelancing panel-based strategies are as given below:

• During the initial phase of the startups, the uncertainties and resources are too limited with no panel of freelancers available. To populate the panel, the freelancer with whom the founder of the startup has previous working relations is outsourced, the work and efforts are being made to motivate him to join the panel. As the time progresses, the referrals and more outsourcings help to continuously enlarge their panels.
• During the stabilization phase, the panel is good enough that could allow access to a diverse pool of freelancers for outsourcing tasks rather than searching freelancers outside. This helps startups to select either crowdsourced panel based strategy and/or non-crowdsourced panel based strategy among the panel freelancers. Crowdsourced strategy involves issuing open call for participation and asking interested freelancers to express their interest by (a) submitting bids, (b) submitting abstract solutions, and (c) submitting complete solutions.
  If option b and c is called competitive crowdsourcing, options a, b, and c are mutually exclusive, and either one is issued for outsourcing the tasks.
  One important remark here is that, in the panel based strategy, the crowdsourcing panel strategy involving competition by submitting a complete solution is also witnessed by startup A. In this mode, the freelancer submits his bid (in response to the open call for participation) by submitting complete solutions, which are selected, and winners awarded. This is because the continuous interaction between panel and startups enhances trust between both parties and increases the motivation of freelancers to contribute towards startup vision. Bidding by submitting abstract solutions is also one of the most exercised options by the startup A.
• During the growth stage, the panel is large enough to allow crowdsourced panel based strategy, which is most cost-efficient. The startup has better maturity with market, expertise with outsourcing processes, and better access to the financial resources, which helps them to attract both abstract and complete solutions from the freelancer panels. The long-term association with freelancers as panel members helps build trust, motivate them towards startup vision, promote their faithfulness toward meeting project objectives, etc. This provides more benefits to startups in the long-term.

The panel based approach may have the situation, where the freelancers may quote higher prices due to their niche and unique skills amongst the panel members that closely matches the task requirements. The startups usually hire from the panel even if the quoted price is marginally higher (perceptual comparison).

There is a strong inverse proportional relation between the prices quoted by freelancers and the size of the panel. However, initially, the panel is smaller, so startups also compare the quoted price with the ongoing market rate (which is identified by the startup founder and is mostly an educated guess). This issue, however, becomes less prominent in the growth phase of the startups. However, the diversely large number of innovation ideas are generated in this freelancing as the startups are benefitted by the competition based freelancing.

Startup B employs the task based strategy, where the freelancers are given association opportunities for the duration of the task (it ends after the completion of the task). No pool of freelancers is maintained that could offer support for the new tasks to be outsourced. Startups could express their willingness to outsource the task through open call (crowdsourced task based) or may simply select the freelancers based on their previous working experiences with them (non-crowdsourced task based). This association could be crowdsourcing based (through the competitive open call through freelancing platforms) or non-crowdsourcing based (invitation based, and, hence, no freelancing platforms are used).

In crowdsourced task based strategy, the freelancing platforms are generally used to outsource the tasks to the crowd of freelancers. In crowdsourced freelancing, there could be two ways of hiring freelancers. Both ways involve an open call for inviting expression of interest from the freelancers. In the first option, the startup publishes their project tasks on freelancing platforms, interested freelancers express their interest by bidding for the project, startup team screens the freelancer bids, interact with bidders through chat/video interactions, selects suitable freelancer, coordinates the work, and finally rewards them (termed as individual bids against open call). There could be agreements about payments, milestones, and deadlines before freelancers are awarded the task.

However, in the second option, the competitive environment among the interested parties is simulated. In other words, the startup team issues the open call for the task, interested freelancers submit the abstract or complete solutions to the problem to be solved, and the best solution is rewarded. If the solution is abstract, then the winner is allotted the task, which, after successfully completing it, gets the agreed payment. Inviting abstract solutions help startups in decision making, concerning the ability of freelancers to take up tasks. However, it depends on deadlines and resources that the startup has in order to analyze plenty of information contained in the invited bids.

In case of complete solutions, the winner gets the payment, as announced in the open call. Once the task is complete, the professional relationship ends. Thus, the startups could manage the outsourcing as a competition (asking for solutions and selecting the best abstract/complete solution) or non-competition (freelancers submit bids, which are screened, leading to selection decisions).

In non-crowdsourced task based, startups do not issue open call for participation. The freelancer, who is easily accessible to startups, or those in professional relationships with the startup team, is invited to execute outsourced tasks (termed as individual pick without open call). These freelancers may be given details about projects, expectations, and price, which are then negotiated with them. Otherwise, the invited freelancer is asked to submit their willingness in outsourcing proposal by submitting the bids and their credentials. They are selected based on their credentials and bids (may involve negotiations). However, this strategy is used when the deadline is strict, as it incurs higher costs, which are not very favorable for startups.

The advantage quoted by the founder is that the crowdsourcing task based strategy avoids the monopolistic attitude of the freelancer. This is because, in case of the non-crowdsourced task based freelancing model, the price charged by the freelancer (for previous outsourced task) is perceptually considered as the base price by the same freelancer for the new task as well (if startup approaches the same freelancer for the new task as well), which may prove costly to startups. If the deadline of the task is very short, and it is hard to find a good freelancer by open call, then the founder usually hires someone known to them. This case arises because of higher levels of uncertainties and risks in the startup ecosystem.

The variation of the selection of the different freelancing task based strategies are given below:

• Initially the startups (startup phase) use either the crowdsourcing task based strategy or non-crowdsourced task based strategy. The startup founder accesses the freelancing platforms for outsourcing the task. Uncertainties and mistakes may require the task to be executed immediately, so a non-crowdsourcing strategy is used under these circumstances.
• As time progresses, the maturity with the market increases and the product reaches the product/market fit, the usual approach becomes a crowdsourcing task based strategy. However, if organized as competitions, then it involves open call for abstract solutions only (rather complete solutions). The reasons for not being able to attract complete solutions are pricing related issues, complexity of the outsourced tasks, lower brand image of the startup, trust issues, etc.
• During the growth phase, the outsourcing is crowdsourcing based only. The open call for participation invites interested freelancers to express their interest by submitting simple bids or abstract solutions. Submission of complete solutions is very rare because the tasks usually outsourced are complex programming related tasks. However, for less complex tasks, such as marketing design material, logo design, business card designing, bug fixing, etc., the complete solutions are possible as the startups have better market visibility (and, hence, brand name) and better resources to support good rewards. Startups could benefit from complete solution based crowdsourcing. However, less professional interaction with freelancers and low trust between two parties (which is better in panel based freelancing model), limit their applicability for complex tasks like programming. This is because freelancers will not like to invest in effortful competitive activity where the outcome is a win or lose binary condition. They will like to contribute their effort only for the tasks which are awarded to them.

Startup C employs a hybrid approach involving the two freelancing association strategies as practiced in startup A and B, i.e., panel based strategy and task based strategy, but marginally modified. Both modified versions are executed in a crowdsourced manner, i.e., by issuing an open call for freelancer participation. The conditional panel based strategy is executed with open call only during the startup life cycle phase, otherwise the strategies are always executed through open call only.

The panel based strategy means that the startup team will only outsource tasks to the panel maintained by the startup issuing an open call for participation, but having the maximum price limits mentioned in the call, which could be rewarded to the successful freelancer after execution of the freelancing task. This is termed as a conditional panel based strategy. Price limit is set in accordance with the judgment of the startup team number regarding the price worth of the task. This limit is derived by considering the number of hours required to complete the task (as believed by startup experts), which is dependent on increment level, complexity of already available code, number of software artefacts required for understanding the system, complexity of work measured in the number of expected lines of code to be coded, etc.

The hybrid approach, thus, does away with the activity that incurs the efforts to negotiate with the freelancers about the pricing issues. Thus, the startup outsources the task to the panel of freelancers using open call, with restrictions on the prices as the rewards.

The startup also executed a task based approach to overcome the issues with non-availability of suitable bids from conditional panel based strategy. This may happen if the prices quoted by the panel is higher (comparative analysis of bid prices and maximum price limits established in open call), the bids collected against open call from freelancing platforms (through task based crowdsourcing model) are processed and suitable a freelancer is selected. Thus, parallelly, the company approaches a crowd of freelancers through open calls on various freelancing platforms (task based approach). The bids that are collected on freelancing platforms are only analyzed if they fail to select freelancers from the panel. These bids act as a backup in case the hybrid approach yields non-optimal bids that satisfy the imposed conditions. The freelancers are motivated to be part of the panel in exchange for the various future benefits. Since the approach involves a panel based strategy (conditional) and task based approach, it is termed as a hybrid approach.

The variation of the selection of the different freelancing hybrid strategies are as given below:

• During the startup life cycle phase, the task based freelancing strategy outweighs conditional panel based strategy because of lack of access to freelancers in professional networks, financing issues, low brand name, etc. However, the situation may allow the availability of good freelancers (in professional networks) suitable for tasks to be outsourced within cost (for instance, student intern). Execution of a task based approach helps to get access to freelancers that could be convinced to be part of the startup panel.
• During the stabilization phase, the panel has a good number of freelancers, so open calls could be issued among panel members with pricing restrictions. In this phase, the restrictions are quite flexible i.e., higher price could be offered depending on the task complexity and resource gaps in startups. Moreover, the task based approach is executed on freelancing platforms using an open call and bids analyzed, if no satisfactory selection is made from the panel.
• During the growth phase, the conditional panel based strategy is more restrictive (beneficial for startups) and the task based strategy is parallelly executed. The bid with higher quality outcome expectations and lower cost is selected. Freelancers are motivated to be part of the panel.

The practices of the studied startups could be mentioned in an abstract manner in

Table 4. Start-up strategies varied across startup life cycle stages.

Start-Up	Start-Up Stage			Remarks
	Stage 1 (Startup)	Stage 2 (Stabilization)	Stage 3 (Growth)
A (Panel Based)	Non-crowdsourced panel based strategy (Individual pick without open call)	Crowdsourced panel based strategy (Abstract solutions). (Individual bids against open call) (Complete solutions) Non-crowdsourced panel based strategy (Individual pick from panel without open call)	Crowdsourced panel based strategy (both complete and abstract solutions)	Initiated with non-crowdsource version (without open call) and finally evolved into crowdsourced version. The long-term association with panel members is beneficial as it enhances trust, motivation level, gearing freelancers towards common vision and objectives and better cooperation.
B (Task Based)	Non-crowdsourced task based strategy (Individual pick without open call) and Crowdsourced task based strategy (Abstract solutions). (Individual bids against open call)	Crowdsourced task based strategy (Abstract solutions). (Individual bids against open call) (Complete solutions are very unlikely).	Crowdsourced Task Based strategy (Abstract solutions for complex tasks) (Complete solutions for less complex tasks). (Individual bids against open call)	Non-crowdsourcing model are only executed under exceptional situations. The strategy is purely crowdsourced. Long-term relationships are less evident.
C (Hybrid)	Hybrid strategy (Individual pick without open call) and Task based approach (using open call)	Hybrid strategy Conditional panel based strategy (Flexible price restriction) + Task based strategy.	Hybrid strategy Conditional panel based strategy (Rigid price restriction) + Task based strategy.	Varied execution of the two approaches as per startup life cycle. Crowdsourced strategy is usually employed. Task based strategy offers backup and as source for enhancing the panel size.

. This table also mentions the common reasons for the observed patterns.

The following are the main points related to the freelancing strategies employed by the startups:

• The focus of all startups is to harness the power of crowdsourcing for outsourcing the task to the freelancers.
• Task based strategy could supplement the conditional panel based strategy by helping the panel grow with freelancers having diverse expertise.
• Lack of resources and higher uncertainties may limit the crowdsourcing based freelancing strategy initially, but as the situation improves, it could offer promising opportunities for the same. The freelancing could start with the freelancer known to the startup team on basis of their previous working experience of those in their professional networks (termed as individual pick without open call).
• Maintaining a panel of freelancers helps to foster innovation, as these freelancers are well adapted to the business environment of the startup due to the strong professional relationships maintained over the time periods.
• Competitive mode of crowdsourcing using complete solutions is possible in panel based freelancing models, which is limited to less complex tasks in task based freelancing.
• Use of crowdsourcing based freelancing helps to foster innovation as diverse ideas are generated from the freelancers submitting abstract or complete solutions as their bids against the open call for participation.

The studied startups do not employ exactly the crowdsourcing models as proposed in [27], i.e., peer production, competition, microtasking. The reasons for implementing these models with flexibility in the freelancing strategies of the startups is explained below:

(a)

Peer production

○

Freelancers do not agree to jointly contribute with other freelancers towards the solution for the outsourced problem. They do not agree to work for free.

○

Startups do not have the resources to offer freelancers enough non-monetary motivation for joint collaborative work.

○

If incentives are to be decided, then freelancers may not agree to jointly contribute towards the novel solution in exchange for “tiny share of incentives”. Moreover, startups will have difficulty deciding the share of incentives among contributors.

○

However, university students could be motivated for such peer production, as offers for industrial trainings. This neither works for very novel problems nor in situations where employees are limited (or employee with the skills to handle the particular problem at hand may not be part of the team).

(b)

Competition

○

The startups usually work with the competition model in later stages of startup life cycles where market characteristics are somewhat understood.

○

It works with all panels, task, and hybrid approaches.

○

The model is employed by inviting bids and complete solutions to select the best solutions for reward.

○

It is executed as a partial model by inviting abstract solutions along with bids to hire the best freelancer.

(c)

Microtasking

○

Startups works under tight schedules and under high uncertainties. This limits the idea of investing resources to divide the tasks into micro tasks.

○

Startups do not have the human resources with expertise to accurately divide the tasks into micro tasks.

○

Startups usually assign the micro tasks to university students, who are associated as internship students.

○

Those tasks are assigned to freelancers, which is complex and difficult for start-up teams that do not have expertise. This limits ideas for microtasking when there is no expertise to decide self-contained tasks.

RQ. 2

What are the issues and challenges faced in establishing and managing the associations?

All of the startups raised several issues and challenges in managing the tasks of freelancer selection. These challenges are mentioned by the sources of data in studied cases. The qualitative information was coded, and codes were merged to result in the below mentioned propositions. This task was challenging as the propositions extracted after analysis of qualitative text of one source of data was compared with another source to provide a test of the already generated propositions. The new propositions either support the old propositions, or they contradict them, which leads to their modification. The final propositions are given below:

• The case for employing a freelancer is purely stochastic. It is hard to predetermine when their services are required, which work will be assigned, and what the scope of work will be. (Probabilistic Arrival).
• The decision to take the services of a freelancer, sometimes demotivates the existing employees. (Employee Demotivation).
• The perceptual value of acceptable bid is purely a subjective judgement. (Price Estimation).
• A lot of effort is made to select the freelancer and a small mistake in selection may be deadly for survival. (Selection).
• The quality of work must be accessed against what was promised and against ability to evolve the system. (Quality Evaluation).
• The previous ratings, work undertaken, trustworthiness of feedback, etc., require a lot of analysis, which is a manual process. (Selection decision aspects analysis).
• Sometimes it takes time to coordinate, communicate, and collaborate with freelancer during the execution of the work. (3C).
• There is difficulty in formulating task descriptions. This happens when the startup team lacks technical expertise in tasks to be outsourced or lack clear understanding of the problem to be outsourced, which is reflected in poorly formulated tasks, with inaccurate deadlines and milestones. (Task Description).
• It is sometimes hard to trust the freelancers. As per the founder of the startup B, “It is hard to trust the freelancer not only from his capability to produce quality work but also if will not quite in later stages. In later case, the startup team is in great difficulty due to the unprofessional and unethical attitude of such freelancers”. (Trust).
• Sometimes it is very hard to attract good and experienced freelancers due to less brand names and low finances available. (Attracting Freelancer).
  For instance, as per the founder of startup A, “The freelancer panel usually consists of Undergraduate, Postgraduate and doctoral students as they are motivated to gain exposure in handling industrial projects, gaining intrinsic satisfaction in executing the challenging tasks, strengthening the future job prospects and to earn money for their livelihood”.
  As per the founder of startup B, “It is very challenging to attract quality freelancers because there is the freelancer quality and outsourcing price varies inversely. Of course, you can negotiate price but tight time to market pressure and limited financial resources inhibit the negotiations, which ultimately results in small compromise with the selection of the experienced freelancers”.
• There is the difference in the working styles of freelancers and startups. For Instance, startups usually like to get work done in less time (to meet tight market delivery schedules). Thus, they impose tight timing pressure on the freelancers with the focus on work related frequent feedbacks. Freelancers usually like to work with full freedom, and this sometimes contradicts with the startup working culture (based on agility and focus on tight delivery schedules), leading to frustration. (Working style differences).

Overall, there are numerous issues and challenges associated, not only with the identification of tasks to be outsourced, but also the selection of good freelancers under resource limitations. These challenges and issues are prevalent in all the studied startups, but differences in the association strategies simulates their differential impacts on them. The varying impacts of these issues and challenges for each studied startup are highlighted in Figure 2, using the ratings scale of 1 to 3, where 1 means low impact and 3 means higher impact.

Figure 2 highlights the varying level of impacts of the issues and challenges across the three startups, post startup phase (i.e., stabilization and growth phase). Startup A, which employs panel based association, seems to be less impacted compared to other two startups. This is because the post startup phase, the size of the panel grows, and the panel freelancers behave virtually like employees of the startups, thereby, continuously motivated for long-term associations.

As per the founder of startup A, “As the time progresses, the panel freelancers exhibit startup culture and work practices, which not only evolve the startup best practices but also make their integration in the startup ecosystem possible”. In other words, the freelancers help the startup to improve their practices, and also motivate them to adapt the other best practices adopted by the startup. The team of freelancers and the employees establish mutual trust and coordination, which lowers the impact of associations challenges.

Startup C, which employs a hybrid association, is more impacted compared to startup A, but less compared with startup B. The use of panel based association in a hybrid strategy provides positive impacts and overweighs the impacts arising due to crowdsourcing based associations (non-panel based). However, the minimal impacts may have strong business impacts on the startups.

RQ. 3

What impact do the associations have on the software development metrics of software startups?

Startups argues that there exist positive impacts of freelancers for the software metrics. Usually if the work is an average task and a good freelancer is assigned to it, then it improves the cost, time, and quality of software. The studied startups reported that freelancing allowed them to reduce cost and time, which otherwise would have been much higher if it had been executed in-house. This is based on the probability of getting an expert and committed freelancer. Moreover, it helps employees to learn how work is done, and continuous interactions with the freelancers, and help them improve their limitations. This may prove beneficial in the long run if similar work is to be undertaken.

However, if a freelancer is not good enough, then, in the long run, it may be difficult to carry out the evolution of the software due to bad quality delivered by the freelancer (and associated technical debt). This may also impact the cost bids by future freelancers. One of the interesting observations from startup A was that they assigned the work to update design documents to freelancers. This work took 3 days with a cost of €73 for the entire work. On average, the employee would have cost the startup around €210 for 3 days, and an outsourcing agency would have cost the startup €700 for the entire work.

The reason for the response of the startups without sufficient quantitative figures is that it is hard to quantify the savings they did for all of the decisions made, related to hiring the freelancers, rather than opting for other options, such as outsourcing it to third parties or hiring full-time employees. Further, the startups were in the mid of their growth phase—not yet turned into companies (i.e., they were yet to get full measurements of the growth and market shares).

4.2. Survey Results

The survey with 54 freelancers is highlighted for each research question below. The 54 freelancers include 21 freelancers who were working with the startups (termed as F1) and 33 who preferred not to work with startups, but in the past, had some associations with them (Termed as F2).

RQ. 1

What strategies do the software startups execute for establishing the association with the freelancers?

The participants reported that the startups usually hired them for undertaking tasks or as unpaid board members (such as panel members). Being private entities, they sometimes requested them to undertake the work, which had pleasant pricing and easy negotiations, without any competitive call. However, this came with tight schedules and high quality requirements (and acceptance criteria). Furthermore, freelancers agreed that they would provide high value proposition and quality services to remain in the freelancing business as freelancing is a two-sided market.

RQ. 2

What are the issues and challenges faced in establishing and managing the associations?

The set F1 reported that they were paid a low amount, and that it should be, because the work required long man hours and the earlier work was done in a quick manner. This made its execution difficult. Documentations are usually what founders tell them, which makes it even harder to understand the changes or additions to be made. The startups have tight deadlines. It is hard for them to decide about association because they never heard of the names of requesting startups and, hence the money to be paid in return for their services is probabilistic. This was the views of freelancers (F2).

In many situations, the estimate of the price quoted in the bids is only based on subjective judgement derived from previous experience. They also reported that, sometimes, they would have to accept work at lower prices because they did not have any assigned tasks at that time and it may have led to exploitation. The backlog of tasks to be executed (now and in the future) is probabilistic and leads to an irregular flow of income. In fact, they keep them extra busy, as well as completely free, which is, actually, resource misuse.

In some cases, startup teams use ambiguous terminology, which results in poor work by freelancers. They then retake rework to satisfy them and get their payments, which leads to waste. This problem usually does not occur when the work to be assigned to the freelancer matches the expertise of the founder, who is able to express what needs to be done and what is expected.

As per F2, undertaking an assignment from startups is a losing deal for high quality freelancers, as to win the competition, they must quote a lower price, and, in the meantime, they may lose any other opportunity that may be higher. However, if a freelancer does not have any assigned tasks to do, for long time, they are forced to consider the associations with startups.

Furthermore, new freelancers may benefit, as they may have higher chances of getting employment in the startups.

RQ. 3

What impact do the associations have on the software development metrics of software startups?

As per F1, undertaking the work at low cost is a saving for the startup, which is reflected in their development costs, tight schedules, and lower development time. However, as per F2, in the long run, the cost and time will increase as low cost development results in technical debt, which is costlier later.

4.3. Comparative Results

The comparative analysis of the findings of the case study and the survey, are compared in this section and presented in

Table 5. Comparative Analysis.

S. No.	Research Question	Case Study Key Finding	Survey Key Finding	Matched
1.	RQ. 1 What strategies do the software startups execute for establishing the associations with the freelancers?	Panel Based. Task Based. Hybrid.	Panel Based. Task Based.	Yes.
2.	RQ. 2 What are the issues and challenges faced in establishing and managing the associations?	Identification of tasks to be assigned, the selection of the best freelancer under the tight resource limitations.	Lower freelancing fees, tight delivery schedules, poor software quality (and other artefacts making software related work harder), ambiguous terminology in task description, less startup related information (affecting trust related issues), chances of employment with company, irregular assignments of tasks, and subjective estimation of the prices of the tasks, are key findings.	Yes (common views). However, results indicate the views from two different entities and, hence, they converge into elaborated meaningful conclusions.
3.	RQ. 3 What impact the associations have on the software development metrics of software startups?	Cost, time, and quality of software is improved if good freelancer is available.	Cost, time, and quality are improved, which depends on the quality of freelancer.	Yes.

.

The results indicate that the freelancers are assigned the work when it emerges, i.e., it is probabilistic to determine when the startup will have assignments for them. The characteristics of the startup puts tight limitations on freelancers to achieve higher satisfaction of the startup team. Such limitations, such as quality issues associated with already developed software artefacts, tight market released schedule, low financial capabilities, difficulty to attract the freelancers (due to brand issues), and limited local market access of these startups (since they are growing in markets) makes it hard to attract the best experience freelancers.

However, if freelancers and startups can keep a high level of trust, then they could maintain healthy relationships, which could be fruitful for both parties. The results help researchers to deduct the following recommendations to create points of synergies between them. Possible recommendations are mentioned in following paragraphs.

To minimize cost, startups could consider having a good recruitment system for freelancers, such as any other employees. Freelancers could be given opportunities to grow with the startup and to avoid monopolist pricing, the panel of freelancers must grow, thereby making the pricing perfectly competitive. The associations with the freelancers must be taken as an opportunity for growth of the employee. Employees must be given opportunity to work with freelancers to learn new things, which could be useful in the long-term.

There is a need to make optimal decision, which otherwise may lead to “Freelancer Selection debt”, i.e., require rework to improve mistakes made as a result of wrong selection of the freelancers. This could invite another type of debt, too. A recommendation system, helping the startup team to make optimal decisions in the selection of the freelancer, assignment of work, and analysis of quality standards, could be very interesting. For startups, they must either reply on known freelancers or must undertake an assessment of competitive proposals (along with bids) to select the freelancer. This task could be made less effortful if ratings and skills could be trusted.

A system will be interesting that could predetermine which task has a higher chance of being allotted to the freelancer. Startups could have a mini parallel type development, where freelancers may be asked to develop the same system, being developed in-house but with lower focus on quality. This redundancy may reduce the chances of the heavy reliance on freelancers. However, this is suitable if the cost of parallel development is offset by the advantages it brings as reductions. This scenario is suitable if the startup has less employees, with multiple roles and, thus, is effected if any one employee leaves.

Moreover, an opportunity must be identified where the real employees of the startup work closely with the virtual employees, and suitable compensations are identified. This means that virtual employees are responsible for the work they undertake with the startup, but not legally under obligations to serve only them. They may take multiple assignments without affecting the quality and signing mandatory declarations for non-disclosure of project information.

The human resources policies involving e-recruitment, compensations, and training could be an interesting area to improve freelancer and startup associations. Startups could employ the suitable machine learning techniques to learn through previous selected bid price datasets for given tasks, with suitable knowledge, skills, and complexity values, to identify benchmark prices for future tasks, which until the date, are based on subjective judgement of the founders (perceptual prices).

Another interesting work could be analyzing the application of the technologies, such as block chains, to make the recognition of freelancers more trustworthy. The startups could also identify the suitable effortless methods to recruit and compensate freelancers that would be cost efficient in the future.

5. Categorization of Freelancing Models

Employing a freelancing model depends on the startup life cycle stage. For instance, during the initial stage of the startup lifecycle, the majority of startups employ non-crowdsourcing approaches (selection of individual freelancer). As the life cycle stages passes with the time, freelancing models become more complex, with a focus on involving crowds to better improve quality and better price estimates.

However, this categorization had been arrived as a result of comparative analysis. The categorization is flexible and, hence, startup could innovate the categories, depending on resources, strategies, product category, founder expertise, and long-term vision of the startup. For example, a startup could optimize the hybrid model (employing both crowdsourced and non-crowdsourced models) to better suit their working context, uncertainties, and resource availabilities. The freelancer model categorization based on comparative empirical analysis is given in Figure 3.

Figure 3 provides helps the startup founder to employ suitable freelancing association strategies according to their context, their requirements, and long-term vision. For instance, if the startup strategy is to be very innovative throughout its life cycle, then it must consider maintaining a panel of expert freelancers and have a good reputation with outside panel freelancers. To enlarge the number of creative ideas (through competitions), the startups should use panel based freelancing because this association helps startups in their late life cycle to attract competition based freelancing associations.

Under such situations, the hybrid approach will be suitable. Thus, startups should focus on building panels and continuously search outside panels for better solutions, to foster innovations. Further, the challenges and issues as identified by this research study could provide startups necessary caution points while trying to use freelancers for innovations and problem solving tasks.

Business impact is a good motivation for them to consider freelancers over the options to hire full-time employees and outsource to third party agencies. However, the ability of startups to manage freelancer associations could be an important factor in evaluating the two options, i.e., freelancers and full time employees.

6. Result Implications

The studied startups managed to conduct their software development activities by involving freelancers continuously throughout their life cycle. The long-term associations helped them overcome the innovation inhibitors catalyzed by the lack of resources. There could be multiple ways of maintaining such long-term relations (i.e., panel based, task based, and hybrid strategies), but all strategies converge into crowdsourcing based versions with the growth of the startups. Crowdsourcing options help them have access to diverse options of the solutions and the freelancers, which, if followed by rational selection decisions, could be a boon for the startups. The freelancer associations bring different challenges, which must be carefully managed to make outsourcing add economic value (product value minus cost) compared to “activities undertaken in house” decisions.

Startups have small team sizes and limited resources (especially financial resources), which makes it hard for them to meet tight market release deadlines. This is further complicated by the necessity to perform continuous experimentations in the market to release the product driven by validated market facts. This means that every interaction with the market is a validated learning, which could be helpful for the future product market releases. This necessitates the long-term associations with human resources (employees and freelancers) to incorporate their learning in future software developments. Thus, freelancer involvement will help startups bridge their skill gaps, and outweigh resource limitations in a cost effective manner. However, maintaining long-term associations is an effortful exercise for startups.

Panel based associations seems to be helpful in simulating the virtual team environment based on mutual trust in the startup, and is quite helpful to minimize the association challenges. However, maintaining panel based freelancers is effortful and costly. This association is beneficial if freelancers are to be involved continuously, for long-term, to outweigh the cost incurred in maintaining the panel. Task based associations seem to be impacted by association challenges, but is suitable for outsourcing tasks that are highly complex for startups to undertake, and are often perceived by them as one-time tasks. Conditional associations incorporate the benefits of panel based and task based freelancing, but executing two associations parallelly incurs the extra effort. This strategy makes sense when there is continuous, long-term involvement of the freelancers, and the tasks are complex as well as costly to perform in-house.

The three strategies (i.e., panel based, task based, and hybrid strategies) could prove to be a reason for the success for startups, but the overall long-term impacts is driven by their abilities to make informed decisions about such associations. For instance, startups must identify the tasks that require shorter-term associations, and those that require long-term continuous associations. For long-term tasks, startups could formulate and execute strategies to implement panels and conditional associations.

The ability of startups to handle freelancer challenges strongly depend on previous software development experience of startup teams, market understanding of the startup team, understanding of the tasks to be outsourced (driven by technical and domain expertise), inclination to offer value to the freelancers counterbalancing the business objectives, ability to establish a friendly working environment for the virtual team, and so on. These factors help the startup team to effectively undertake and execute the outsourcing decisions, which provide joint value to both the startup and the freelancers.

The impact of the freelancer involvement on software development metrics compared to the impact of undertaking it in house is always an estimation, and not an accurate calculation. This is because quantitative measurements of the impact is hard to quantify, and is always a probabilistic estimation. Further, business decisions are usually taken by startups in an informal manner and are less documented. This makes it even harder to map the decisions to the savings by considering the prevailing market rates at the time of the decisions. However, the estimations suggest the positive impact of the freelancer involvement on the overall software project metrics that could help startups improve their success rates. However, it will take more time for startups that are converted into companies to report actual benefits ripped by freelancer involvement. The long-term, for a more valid impact, is predicted, because currently startups have high failure rates, and freelancers are usually not involved for longer terms by startups.

7. Result Assessment

Member checking is done to access the validity of the results. Efforts are being made to see if the startup team and freelancer responses are well interpreted by the researchers, and no new information is left to be shared with the researchers, which could enhance value of the study results. A questionnaire containing a list of eight closed questions and one open question was circulated to the research study participants. They were asked to rate each question using a scale of 1 (strongly disagree) to 5 (strongly agree). The rating of the questions, representing the practices of other startups (question number 1 to 6), could be an issue for the raters of another startup (where the practice is not practiced); thus, the questions were formulated in a manner that does not represent the rating of the practice, but rating of how well it could be useful to other startups. The questionnaire is given in Appendix A.

The quantitative responses for each question obtained from startup team members (total 7) and freelancers (total 54) that participated in the research study are recorded in

Table 6. Rating responses from the case study participants.

Question Number	Category	Total Responses (Categorized per Rating)
		1	2	3	4	5
1.	Panel based approach (Non-crowdsourced)	1	2	15	28	15
2.	Panel based approach (crowdsourced)	1	7	23	20	10
3.	Task based approach (crowdsourced)	0	0	5	16	40
4.	Task based approach (Non-crowdsourced)	0	0	0	20	41
5.	Hybrid approach (Conditional panel based)	2	4	19	23	13
6.	Hybrid approach (Task panel based)	1	2	13	27	17
7.	Hybrid approach	0	0	0	14	45
8.	Challenges	0	0	0	10	51
9.	Business impacts	0	0	0	11	50

for further analysis.

Table 6 gives shows that majority of the participants agreed to the results as reported by the case study. However, the following are important observations:

• The questions related to the task based approach (question 3 and 4) are positively skewed, i.e., most ratings are in the highest value of 5 (66%), followed by ratings of 4 (26%) and ratings of 3 (8%). In particular, the raters agreed that the task based approach in a non-crowdsourced manner should be executed under exceptional circumstances, while crowdsourcing one could be made more participatory if the startups could manage long-term relations with the freelancers that associated with them for any task during the time period.
• The questions related to the hybrid approach (question 5 and 6) have wide scattering of the responses, but the majority of raters seem to agree with the potential of the conditional panel approach and task approach. For question number 5, 61% of ratings belong to the rating score of 4 and 5 and 31% to the score of 3. For question number 6, 72% of ratings belong to the rating score of 4 and 5 and 21% to the score of 3. Overall, responses indicate that if the conditional panel approach and task approach are executed effectively, they not only create value to each other, but create synergy to create value to the startup.
• The ability of the hybrid approach to create long-term associations with freelancers (because of the focus on continuously populating freelancer panels) (question 7), the various challenges reported (question 8), and the business impacts of freelancer involvement in startup operations (question 9), are rated highly by the raters.
  The ratings for panel based freelancing have the scores distributed between the rating score of 3, 4, and 5 (with minor ratings of 1 and 2). This seems to be because, on one side, the panel idea seems to be very innovative for the raters, but their concerns about panel management lead to such type of distribution. Although the majority of values reflect the respondents agreeing to the aspects related to panel based freelancing, small confusion seems to be prevalent among them. As per one of the respondents, “Panel based freelancing is another version of task based freelancing except that rather you go to the freelancing platform to talk with “completely unknown” experts, you talk with your large panel with whom you have professional relationships. The idea is really beneficial for startups that require long-term relations without spending too much, yet I really do not know how we manage such a panel when we have nothing to offer, maybe founder expertise and professional relationships could be a good promoter of the idea”.
  One important observation was that the founder reputation in the market (for instance, as former manager of a reputed company) could motivate freelancers to be part of a panel driven by enhanced trust and foresight of bright future aspects.

Overall, the responses indicate a higher agreement of the majority regarding the results reported by the case study. Panel based freelancing has the potential to overcome challenges with freelancing associations and provide value to the startups, yet maintaining such a panel is quite effortful.

8. Study Limitations

There are some marginal limiting aspects of the study that provides a roadmap for future work. The findings, as reported by the case study, are based on the perspectives given by the startups and the freelancers, along with those observed by the authors in the startup working environment. This however could be further validated by accessing the software documentations and business records, which were not maintained by the startups, as their focus was to invest in activities that result in product/market fit. However, non-accessibility to the documentations and business records is not a major threat due to involvement of multiple data collection methods and multiple cases.

The startup team members who participated in the study were selected by the selected startup only, which may have resulted in the meaningful perspectives to be missed (for instance, key startup employees with unique perspectives), which, could otherwise have provided more insight. However, this threat of missing some perspectives is overcome by observations and freelancer involvement (including the negative cases).

The generalizability of the case study, based on the study of three startups only, is negligibly (or, maximum, marginally) impacted, because the number of startups that are successful in the market and have involved freelancers for longer terms, are too limited. The exceptions could be those startups that do not declare their dependency on the freelancers (maybe due to financing relating issues) and, hence, are not included in this study, but their number could be almost negligible. Hence, the population of successful startups that involved freelancers for longer terms are very limited, which signifies that the sample size (three startups) is a true estimator of the population, which makes the study generalizable at present.

The freelancer’s long-term involvement in software startups is still in its infancy stage, and will take a long time to capture a matured state. The infancy state and limited number of such startups made it difficult to perform the quantitative study of the freelancer association impacts on the software development metrics (and, hence, the overall business). It will take longer for startups to enhance their market success rates and report freelancer association impacts, based on long-term estimations.

9. Conclusions and Future Work

The finding of the paper reveal that startups face more challenges in outsourcing their work to freelancers. This is because of a lack of resources to attract good freelancers and the inability to create valuable outsourcing offers that are perceived as valuable by freelancers. The freelancers find it difficult to contribute to the completion of the task at lower rates, tight delivery conditions, and accumulated debt. This may activate the situation where the work is accomplished by less experienced freelancers (for example, a less experienced freelancer or a degree student) that further complicates the future associations with experienced freelancers.

The startup requires minimal effort techniques to search for good freelancers and predetermination of tasks to be outsourced to manage the uncertainties. The freelancing if undertaken by experienced freelancers is reflected in improved software development metrics. However, there is much more to be done to make their relations long-term and fruitful, such as establishment of a system of joint growth, trust, and virtual development teams to make the entire process a learning one. Fostering innovation in startups is, thus, a trade-off situation, which is limited and supported by many conflicting parameters.

The case study is able to generate and test few propositions/hypotheses related to involving freelancers in software engineering activities of the startups. This includes the following:

• The panel based freelancing initiates from the individual freelancer picks, to well managed crowdsourcing based selections, depending on the availability of resources, which depends on the life cycle stage of the startup. However, this freelancing helps to develop long-term trustworthy relations with the freelancers, and is the source of diversely large innovation ideas.
• Task based freelancing could start with either individual pick or crowdsourcing through freelancing platforms. However, this freelancing could not attract diversely large numbers of innovation ideas due to less participation of freelancers in competitive crowdsourcing for complex problem solving tasks.
• All types of freelancing converge into crowdsourced based freelancing, whether done with panels or with outside panel experts.
• The startup could maintain a portfolio based freelancing association approach, where the individual strategy could be executed, or strategies could be adopted to specific contexts and merged into single unified strategies.
• Startups have difficulty in managing the freelancing process due to its probabilistic nature, resource limitations, and their newness in the market.
• Association with a good freelancer from the beginning of the project affects future long-term associations. Long-term associations are beneficial for startup growth.
• Informed decisions about freelancing process canvas (for instance, tasks to be outsourced, perceived pricing, duration, etc.) will support active participation from freelancers. This is a two-sided market, hence, both startups and freelancers have to provide innovative value proposition to each other to establish and maintain long-term relationships.

In the future, it is expected that the results of the study that provide a holistic view of the freelancer associations for software startups could be fine grained to their involvement across individual software engineering activities. This will be quite helpful to the startup community and researchers, as different software engineering activities provide different challenges for freelancer involvement due to different skill requirements, and the challenges unique to individual activities. For instance, requiring engineering (or value proposition innovation) requires establishing long-term relationships, undertaking the continuous rework to incorporate continuous learning, freelancer and startup team perspectives merging, and so on [4], over and above the generic association challenges.

Literature has limited ability to provide rich, empirical support for freelancer supported software engineering at present [6]. In the future, the rigorous research in freelancer involvement in startups across individual software engineering activities will help provide rich, empirical evidence to the startup community, to foster rational decision making. The research domain is still in its infancy stage, but recent attraction of researchers will help with advancing to the maturity stage in the near future. The results reported by this case study provide rich opportunities for future extensions, as per the dynamism taking place in micro and macro environments, and their unique adoption by startups.