INTRODUCTION

The world economy is currently characterized by a high rate of innovation, in which science, technology, and information and knowledge play a key role in business competitiveness (Chabowski and Mena, 2017; Comin, 2016). The phenomenon of entrepreneurship is of vital importance for economic development since the emergence of new start-ups leads to the generation of new jobs, especially skilled employment. In this sense, the entrepreneur is a key agent in the society and he/she contributes to economic growth and social development. Additionally, Bergset and Ficher (2015) state entrepreneurship activity is turning economies more dynamic and sustainable.

The younger generations are attracted by the philosophy of creating their own business. All over the world, with special emphasis on countries such as the USA, India, the United Kingdom, and Israel, governments and universities have invested significantly in boosting entrepreneurship (Ács et al., 2018). In this context, several start-ups have emerged all over the world. These businesses are characterized by the appeal to continuous innovation, reduced number of employees, highly dynamic and open environment, relatively short product lifetime, and high potential for value creation (Ries, 2011; Spender et al., 2017).

At the same time, several initiatives have emerged that incentives the emergence of new business ideas promoted by universities, governments and science parks. Many of these initiatives are also sponsored by other companies, especially the larger ones. Ideas competitions are of great importance to young entrepreneurs, as they enable them to obtain seed funding to work and maturing their business ideas. The themes of these business contests can be very heterogeneous and address diverse areas that can be related to health, environment, technological development, among others.

Regardless of the area of the business ideas competition, one of the common elements between them is the existence of a jury that evaluates business ideas in a comparative way, since only a small number of them will be approved and obtain funding and/or other types of support. In the context of these juries, several problems emerge, namely: How to consider the opinion of the various members of the jury? What are the most relevant criteria for assessing each project? How to ensure that the opinion of each jury member is of equal importance?

In fact, the answer to these questions is complex and has not been uniformly resolved. Each competition resolves these issues autonomously and informally. Therefore, there is a need to find a solution that simplifies and supports the evaluation of the various projects in the competition. In this sense, the objective of this study is to propose the development of a Decision Support System (DSS) for business ideas competitions that support the evaluation of a business ideas competition and facilitate the work of juries through the existence of a robust and transparent evaluation mechanism that considers the same evaluation criteria for all projects, the quality and merit of each project and a distinct perception of the member of the jury regarding each project. The adoption of the AHP method allows comparative evaluation of the various competing projects against the criteria defined by the jury president. Saaty (2004) establishes the usefulness of this method in decision problems with multiple attributes and decision agents. Additionally, the use of AHP facilitates the proactive participation of jury members in the decision-making process. This is also a key aspect to increase the transparency of the decision-making process. This study is original and innovative in the field of its application since it was not identified in the literature the use of any DSS that allows the business idea competitions to assist them in the process of managing the several evaluations of juries.

The manuscript is organized as follows: Initially, a literature review is performed in the field of decision support systems and metaheuristics models, with emphasis on the AHP method. After that, the study methodology and the prototype developed are presented. Next, the main results of the project are discussed considering four scenarios in which the importance of each criterion and the evaluation of each member of the jury are changed. Finally, the conclusions of this study are drawn.

LITERATURE REVIEW

The Role and Structure of Decision Support Systems

Decision-making analysis was initially conducted by the Carnegie Institute of Technology in the late 1950s and early 1960s. The Massachusetts Institute of Technology (MIT) applied computer technology to decision-making theory in the 1960s. By the 1980s, intensive research on DSS was underway, and new theories and concepts emerged from single-user models of DSS, including organizational decision support systems (ODSSs), group decision support systems (GDSSs) and executive information systems (EISs). By 1990 DSS was broadened to include data warehousing and online analytical processing (Shim et al., 2002).

DSSs are typically composed of four components (Schwartz, 2015):

• Data management component – it is responsible for managing the data coming from databases or other external files. In this component the inputs and outputs, entities, models, assumptions, variables, and other data should be stored;

• Model management component – it is responsible for the management of knowledge modeling. This component should manage the representation of current and historical models. Consequently, this component should use the DBMS to store the data and should allow the selection, modification, removal, and inclusion of models;

• User interface management component – implements the dialogue and interaction with the decision maker. Turban et al. (2004) argue that human-machine interfaces should be adaptable, flexible, transparent, self-explanatory and efficient;

• Knowledge-based component – supports the knowledge management process. Performing some tasks in a DSS may require specialist knowledge or the use of artificial intelligence modules.

The choice of a DSS shall be fundamentally based on the nature of the activity or the decision to be taken. Sharda et al. (2014) recommend that the decision structure, its complexity, and impact on the organization should be analyzed in its choice. In this sense, a DSS is not always the best option. DSSs should be fundamentally chosen when we are faced with unstructured or semi-structured tasks, with a high degree of complexity and with a significant impact or level of risk for the organization (Karaman & Ghaffarzadeh, 2014).

It is possible to identify a significant number of benefits brought by DSSs. Dulcic et al. (2012) mention the speed of the decision-making process and the increased consistency of the decisions taken. Hilletofth et al. (2016) emphasize the role of DSS in simulating a reality model, which reduces potential losses and risks. Additionally, the DSS facilitates communication between decision-makers and allows access to previously collected and compiled knowledge (Zhengmeng and Haoxiang, 2011). However, DSSs do not make decisions for humans, but only enunciate and provide possible solutions to problems. Therefore, these systems contribute to and support the decision-maker, but do not replace the decision maker.

DSSs can be used in a wide range of application areas, such as design and planning logistics activity (Qaiser et al., 2017), supply chain management (Hassan et al., 2015), diagnostics in the healthcare field (Assena et al., 2013), urban planning (Almeida, et al., 2017), in the process of management educational services (Vohra and Das, 2011), software industry (Farshidi et al., 2018), or in security analysis (Tangil et al., 2014).

Metaheuristics and the AHP Method

One of the basic characteristics of a DSS system is the inclusion of models. The idea consists in the adoption of interactive computer-based systems that help decision makers use data and models to solve unstructured and semi-structured problems. DSSs have evolved during these last years to encompass multi-component systems that include various combinations of simulation modeling, optimization techniques, metaheuristics, and artificial intelligence techniques (Sharda et al., 2014).

A meta-heuristic is a general search strategy that tries to efficiently explore the space of viable solutions to this problem (Gigerenzer et al., 2015). They are approximate algorithms that incorporate mechanisms to avoid confinement to local minimums or maximums. Specific knowledge of the problem can be used in the form of heuristics to assist in the search process. Xu and Zhang (2014) advocate that the strategies for implementing a meta-heuristics should be chosen in such a way that there is a dynamic balance between exploitation and exploration. Exploitation seeks to consolidate research in target zones of the solution space, while exploration seeks to cover the solution space as much as possible in order to find better solutions to the problem. Several stopping criteria can be defined, such as (Asih et al., 2017): maximum execution time; maximum number of iterations; finding a sufficiently good solution; and a maximum number of iterations without improvement.

The AHP is due to Saaty (1980) and is often referred as the Saaty method. It helps structure the decision-makers’ thoughts and can help in organizing and structuring the problem in a manner that is simple to follow and analyze. Basically, the AHP helps in structuring the complexity, measurement, and synthesis of rankings. The AHP has proved a theoretically sound and market-tested and accepted methodology in a wide variety of domains, such as resource allocations, forecasting, business process re-engineering, balanced scorecard, benchmarking, public policy decisions, healthcare, and many more (Bhushan and Rai, 2004).

The AHP can be decomposed in four steps like it follows (Saaty, 2008):

1. Contextualize and define the domain of the problem;

2. Structure the decision hierarchy from the top with the goal of the decision, then the objectives from a broad perspective, through the intermediate levels (criteria on which subsequent elements depend) to the lowest level (which usually is a set of the alternatives);

3. Construct a set of pairwise comparison matrices. Each element in an upper level is used to compare the elements in the level immediately below with respect to it;

4. Use the priorities obtained from the comparisons to weigh the priorities in the intermediate levels in order to obtain its overall or global priority. A scale between 1 (equal) and 9 (extremely strong) is adopted. Finally, the global priority is established on a scale between 0 and 1.

A basic, but a very reasonable assumption, is that if attribute A is absolutely more important than attribute B and is rated at 9, then B must be absolutely less important than A and is valued at 1/9. These pairwise comparisons are carried out for all factors to be considered, usually not more than 7, and the matrix is completed (Saaty & Vargas, 2012).

AHP helps decision makers find one that best suits their goal and their understanding of the problem. According to Palcic and Lalic (2009), it provides a comprehensive and rational framework for structuring a decision problem, for representing and quantifying its elements, for relating those elements to overall goals, and for evaluating alternative solutions. Once the hierarchy is built, the decision makers systematically evaluate its various elements by comparing them to each other two at a time, with respect to their impact on an element above them in the hierarchy. In making the comparisons, the decision makers can use concrete data about the elements, but they typically use their judgments about the elements’ relative meaning and importance. It is the essence of the AHP that human judgments, and not just the underlying information, can be used in performing the evaluations (Sharma, 2018).

METHODOLOGY

Design

The study is composed of three main phases as depicted in Figure 1. The preliminary stage seeks at an early stage to formulate the theory on the role and structure of decision support systems and metaheuristics. Next, the research questions and the functional and non-functional requirements of the prototype are formulated. In the fieldwork stage, the prototype was implemented and divided among the team members according to the profile of each member (e.g., project manager, software developer, software tester, etc.). Even at this stage, software tests were carried out that included acceptance tests, integration tests, and system tests. Finally, in the evaluative design, the authors sought to answer the research questions previously formulated with reference to the functionalities and tests performed on the prototype, also considering a comparative analysis of the elements identified in the literature. Finally, the conclusions were drawn and future research directions were formulated.

Figure 1. Phases of the adopted methodology

Equally important to evaluate the results of this study was the definition of the research questions. These research questions were formulated with reference to the operating model of the juries of entrepreneurship and business idea competitions.

There are many international, national and local contests for business ideas. MassChallenge is a global start-up competition in the field of technology promoted by MIT. It is open to start-ups and innovative project that present high-growth potential, in which technologies can be patented or not (MC, 2019). The criteria that each jury applies are different according to the business area. Hello Tomorrow Global Challenge (HTGC) is another worldwide competition dedicated to science & technology startups (HTGC, 2019). This competition offers twelve tracks, such as Data and Artificial Intelligence, Global Health, Industry 4.0, or Aeronautics. Contrary to the MC, the HTGC uses the same criteria in all tracks in the evaluation of competing companies. Four criteria are considered: (i) innovation; (ii) potential impact; (iii) leadership; and (iv) viability. Another very popular competition is held in Sweden. The Venture Cup (VC) is the biggest startup competition in Sweden that offers two sub-competitions: (i) Idea Venture Cup (IVC); and (ii) Startup Venture Cup (SVC). IVC is a competition for business ideas in an early stage, while SVC is a competition targeted start-ups in the early phase with low commercialized revenues (VC, 2019). There are specific rules associated with each competition.

This exploratory research on the organization and functioning models of business ideas competitions is relevant to define the research questions of this study. The information gathered from these competitions allows us to conclude that the number of juries of each competition is quite diverse and have multidisciplinary skills, and it is common to find jury members with experience as entrepreneurs. It is still necessary to conclude that the criteria considered in the evaluation should be defined at the time of publication of each competition, but that they can be quite distinct according to the characteristics and operating model of each competition. In this sense, three research questions emerge as relevant:

• RQ1: Do all jury evaluations have the same importance?

• RQ2: Can the evaluation criteria be customized for each competition?

• RQ3: Can the importance of each evaluation criterion be personalized by each member of the jury?

The operating model of each competition is also very heterogeneous. MC develops an accelerator program in which entrepreneurs develop their business over four months. HTGC establishes a model consisting of three phases. In a first phase, the top 500 most promising start-ups are selected. Then, the judging committee chooses the seven best startups per track. In the last phase, the winner of each track is announced. Finally, IVC appoints the national top 20 list after a screening process of the top 50 ideas, while SVC nominates the top 10 startups for regional finals and, after that, the national winner is chosen.

The comparative analysis of the operating model of each competition allows us to realize that it is important to have a business ideas software that is sufficiently comprehensive and customizable to provide an ordered list of ideas in competition, instead of presenting exclusively the winner of each competition. Furthermore, it should allow the identification of tie situations and propose approaches for their mitigation. In this sense, two more research questions were defined:

• RQ4: Is there a ranking of competing projects?

• RQ5: How are any tie situations resolved?

Modeling

The AHP method is used by jury members to evaluate competing projects. Figure 2 illustrates the process of operating the AHP method using three criteria and three competing projects. The first level is the goal of the analysis. In our case, it represents the best business ideas in competition. The goal is not only to show the best business idea in competition but provide an ordered ranking of business ideas. The second level presented the several considered criteria (i.e., price, potential, and product viability). The third level considers alternative choices, which are the different competing projects.

Figure 2. Structure of the AHP method

A decision model was developed. Initially, the AHP method is used by each jury member to evaluate the relative importance of each criterion and project.

The result of this process returns two vectors: (i) one vector with the importance score of each criterion; and (ii) another vector with the importance score of each project. In both vectors, the sum of all elements is equal to 1. Then, the overall priority of the project is obtained by the sum of the multiplication of each criterion with the weight of the priority. Finally, the opinion of all jury members has to be uniformly considered. For this purpose, the overall priority of each project given by each jury member is aggregated and then divided by the number of jury members. Consequently, the elements of this list are ordered decreasingly and disclosed to the jury president.

Physical and Logical Architecture

Figure 3 presents the physical architecture of the application. The software is organized into three layers (i.e., data mapping layer, business logic layer, and presentation layer) and is based on three technologies: (i) C# is a programming language used to develop the software using the Microsoft Visual Studio 2017 framework; (ii) SQL Server, which a Database Management Systems (DBMS) that is used to store data in the application; and (iii) XML and XML Schema (XSD) are used only to export the results of the competitions. Additionally, Windows Forms and Language-Integrated Query (LINQ) are used to enable connection to the database and to interact with the presentation layer (i.e., judge account, DBA account). These languages offer multiple features and libraries that enable the users to interact with Microsoft systems and allow great application interoperability and stability without losing consistency (Jain, 2017).

Figure 3. Physical architecture

Figure 4 presents the logical architecture using a UML component diagram, which shows the hierarchical decomposition of the system into logical modules (packages of classes with interaction among them) and the specification of the interfaces and dependencies between the modules. The entry point of the application is through the class “authentication.cs”, which is responsible for performing the authentication process. This is a central class in the application because all operations require user’s authentication. After that, the user can edit his/her profile, including his/her name, photo, and password. In the main panel, the “competition.cs” class is responsible for checking the state of the competitions considering the current date. It is also possible to access the history of competitions in which the user has participated through the “history_of_competition.cs” class. The “competition.cs” class is also responsible for the creation of a new competition, edition of competitions and inclusion of new projects to existing competition. Finally, the “evaluation_competition.cs” class evaluates the result of each jury using the AHP method and determines the final result of the competition including all the evaluations of the jury members.

Figure 4. Logical architecture

Requirements and Prototype

The functional requirements describe the functionalities and services offered by the system. Sommerville (2015) argues that in the functional requirements all services should be specified and that they should not have contradictory definitions. The prototype implements the following twelve requirements: (i) authentication in the application; (ii) create new competitions; (iii) create a new project; (iv) project evaluation using the AHP method; (v) draw the dashboard; (vi) calculation of statistics; (vii) access to the history; (viii) calculate the final results of the competition; (ix) export results of the competition; (x) edit profile; (xi) create a new user; and (xii) enable/disable the user from the platform. The requirements are mapped in the UML use cases diagram of Figure 5.

Figure 5. Use cases diagram

The non-functional requirements define the properties and restrictions of the system. They can be applied to the whole system or a part of it. Sommerville (2015) considers those non-functional requirements are directly related to user satisfaction and are critical to the success of software engineering projects. The following non-functional requirements were defined: (i) performance, the adopted business intelligence models must be run on any low/medium portable, desktop, or mobile device in a processing time fewer than 5 seconds; (ii) security, SHA cryptography hash functions are adopted to encrypt user’s password; (iii) usability, the interface (e.g., button, text box entries) have a clear indication of their functionality; and (iv) scalability, new juries, criteria, and projects can be added without compromising the application’s performance.

Access to the application is done through the authentication process. The application offers a login system, in which the user must provide his/her username and password, as depicted in Figure 6.

Figure 6. Authentication

After a successful authentication process, the user will have access to the main menu where he/she can consult the list of ongoing competitions or create a new competition. From the moment the user creates a new competition, then he/she becomes the jury president of that competition. This way, and simultaneously, the same user can be jury president of a competition and jury member of other competitions that are established. When creating a new competition, the user must indicate the name of the competition, the number of criteria, the start and end date, data on the projects and promoters. The user also associates a description to each included criterion. At any time the user can access the details of a given competition. Figure 7 shows an example of a competition with 2 criteria already established, but which has not yet been started and there are no defined projects. In this sense, the completed evaluation is still at 0%. Figure 8 complements the interface showed to the user by presenting a diagram of UML activities that describes the process of creating a new competition.

Figure 7. Access to competition’s details

Figure 8. Activity diagram of creating a new competition

A jury member can evaluate a project that is associated with him/her. For this purpose, he/she must comparatively assess the importance of each criterion and the performance of each project in relation to each criterion. This comparison is performed using a listbox in which the user compares each of the criteria and alternatives as shown in Figure 9. The interface is expandable vertically according to the number of alternatives and horizontally according to the number of criteria.

Figure 9. Evaluation of the projects using the AHP method

Finally, after the date set for the insertion of evaluations has been reached or after all jury members have evaluated all projects in a given competition, it becomes possible to access the project dashboard. This dashboard intends to summarize the most relevant aspects of the project and provide the final score of the competition. Figure 8 shows an example of the dashboard for a competition. The dashboard window is divided into two areas: top and bottom. At the top, the user finds information on the number of criteria, number of projects, number of juries, and number of promoters, the percentage of juries that assessed the project, gender of promoters and distribution of projects by categories. At the bottom, the user has a graphical analysis of the promoters’ age distribution and the final classification. The final classification is ordered decreasingly. In case of a tie, the average age of the promoters serves as a tie-breaker, like in the example provided in Figure 10.

Figure 10. Dashboard

ANALYSIS AND DISCUSSION OF RESULTS

Four scenarios were considered to explore the potential of the prototype according to the research questions established in the study. These scenarios intend to change the importance given to each criterion and the impact of the jury members’ evaluation on the serialization of alternatives. The same number of criteria (C=3) and alternatives (A=4) are maintained in all scenarios.

Scenario I - Same Importance of the Criteria and Alternatives

In the first scenario, all criteria are of equal importance and there is no alternative that is better or worse in any of the criteria. Table 1 shows the final score obtained for the total of the alternatives and criteria. The result of the serialization is a potential draw of the alternatives. In this situation, the lowest average age of the promoters is used as the tie-breaker criterion. If the mean age of the promoters of A1 is lower than A2, and in turn is lower than A3 and A4, then the final serialization of the alternatives is A1, A2, A3, and A4.

Scenario II – Distinct Importance of the Criteria

In this scenario, C1 is significantly more important than C2 and C2 is slightly more important than C3. Table 2 shows the obtained results applying the AHP method to scenario II. It is demonstrated that the different importance of the criteria has no impact on the serialization of alternatives. In this sense, and applying the same serialization system based on the average age of the promoters, the final result would be identical to the scenario I.

Scenario III – Distinct Evaluation of Alternatives

The third scenario gives the same importance to all criteria, but in which the evaluation of each alternative is distinct in such a way that: (i) A1 is significantly better than A2 in the criterion C1; (ii) A1 is significantly better than A3 in the criterion C2; and (iii) A2 is slightly better than A3 in the criterion C2. Table 3 shows the final results applying the AHP method. This situation has an impact on the serialization of alternatives. A1 is the best alternative with a final result of 0.3265, followed by A2, A4, and A3. In this sense, a different evaluation of the importance of the alternatives in relation to a given criterion has an impact on the serialization of the alternatives.

Scenario IV – Distinct Evaluation of Alternatives and Criteria

The fourth scenario gives distinct importance to the criteria and alternatives. Basically, it applies the evaluation given to the criteria of scenario II and the evaluation given to the alternatives of scenario III. Looking at Table 4, it is possible to conclude that this situation also proves changes in the serialization of alternatives. The best alternative is A1 with a final result of 0.4229, followed by alternatives A4, A2, and A3.

Inclusion of the Evaluation of all Jury Members

The previously established scenarios consider only the evaluation of a single jury member. Typically, business idea contests have several jury members, and therefore it is relevant to simulate a scenario in which several jury evaluations are considered. In this case, and due to simplicity reasons, it was considered that each of the four previous scenarios corresponds to the evaluation of a different jury member. Table 5 presents the final serialization of a competition considering four jury members. The final score is calculated considering the sum of all jury scores divided by the number of jury members. Finally, the serialization of the competition results is decreasingly ordered. In this sense, the best alternative is A1 with an average equal to 0.3124, followed by A4, A2, and A3.

CONCLUSIONS

Business ideas contests have gained increasing predominance and relevance especially among young highly qualified graduates that intend to launch their own business. The operating model of each competition is quite heterogeneous, but some common elements stand out, such as the need for a jury president to formulate the rules of each competition and manage the functioning of each jury, the existence of several jury members typically with a multidisciplinary profile, and the serialization of the projects in competition considering the evaluation of all jury members.

Looking at the research questions established in this study it was possible to conclude that:

• RQ1 – The application considers all evaluations of the jury members that have the same importance in the serialization of projects in competition;

• RQ2 – The application stores in the database the criteria used in each competition. These criteria can then be reused in other competitions. In each competition it becomes possible to use criteria stored in the database or to customize new criteria that can be subsequently adopted in new competitions;

• RQ3 – Using the AHP method each of the jury members can specify the relative importance of each criterion. This comparison is performed using the Saaty scale;

• RQ4 – All jury evaluations are taken into account in the serialization of the projects. Projects are serialized decreasingly according to their final score;

• RQ5 – The average age of the promoters of each project is used as a tie-breaker in an eventual tie scenario in the serialization of the projects.

This study simultaneously has an impact on the theoretical and practical functioning of an ideas competition. In a theoretical and conceptual approach, this study proposes the use of the AHP method to facilitate and turn more robust the evaluation process of competing projects. From a practical point of view, this study has the potential to be used in several national and international competitions for ideas, regardless of the area of competitions and the number of jury members.

The project is still in the prototype phase. Therefore, it is relevant to test the performance offered by the application in a real scenario. Additionally, as future work, it is intended to include new features in the developed prototype. First of all, it would be interesting to customize the approach to deal with tie situations in the serialization process. At this moment, only the average age of the promoters is considered, but it would be relevant to consider new criteria according to the specificities of each business competition. It is also intended to integrate the process of notification of a new jury or evaluation with the email system. Finally, the authors intend to develop in the future a Web and Mobile application that will improve the application’s portability.