HOME | ÍNDICE POR TÍTULO | NORMAS PUBLICACIÓN

Espacios. Vol. 37 (Nº 24) Año 2016. Pág. 6

Hospital knowledge management: A performance measurement literature analysis

Gestão de conhecimento hospitalar: uma análise de literatura de medição de desempenho

Leonardo ENSSLIN 1; Clarissa Carneiro MUSSI 2; Sandra Rolim ENSSLIN 3; Ademar DUTRA 4; Alexandre Felipe MACHADO 5

Recibido: 18/04/16 • Aprobado: 23/05/2016

Contents

1. Introduction

2. Theoretical framework

3. Research methodology

4. Results found

5. Conclusions

References

1. Introduction

Knowledge Management (KM) consists of creation, acquisition, sharing and use of knowledge to promote organizational performance (WEN, 2009). With the use of KM practices and techniques, it is possible to formalize tacit knowledge, which then allows the continuous improvement of the processes that require the highest level of efficiency to occur (LOTTI, 2014).

In healthcare, Knowledge Management usually takes place in hospitals and clinics and a major concern, having in mind the multiple actors and multiple perceptions of problems in it, is the high level of difficulties present. Thus, these institutions are constantly concerned about the level of maturity at which Knowledge Management takes place. Knowledge Management in hospital institutions promotes the reduction of the life cycle of services and costs, generates higher return on investments, provides a higher level of patient satisfaction as well as the continuous learning process of doctors and other staff members within the organization, aiming at excellence in service provision (COLAUTO & BEUREN, 2003).

In this context, this paper proposes the following research question: How to select a limited set of scientifically relevant articles aligned with performance evaluation of knowledge management in healthcare, and based on this bibliographic portfolio, conduct bibliometrics on the following parameters: journals, articles and authors?

This paper has the purpose of selecting international scientific publications as of the year 2000, which deal with a fraction of the literature on Performance Evaluation of Knowledge Management, making it possible to present, from the perspective of the topic, their most present journals, articles and authors. In order to achieve the purpose of the research, the instrument of intervention called Knowledge Development Process – Constructivist (ProKnow-C) was used, as it is a structured process of literature review with a constructivist perspective; therefore, aligned with the purpose of the research.

This research is important because, based on the identification and analysis of the characteristics of the publications in the fragment of the literature in question, knowledge on the topic is created and can be summarized in guidelines for new research. As for its uniqueness, it is justified by the lack of studies, based on the searched database, about the proposed topic.

In addition to this introductory section, this paper is structured in the following sections: section 2 is composed of its theoretical framework, which is about "Performance Evaluation" and "Performance Evaluation of Knowledge Management"; section 3 introduces the research methodology; section 4 presents results found; and finally section 5 presents the researchers´ conclusions and final considerations as well as references that guide this paper.

2. Theoretical framework

This section is composed of two parts. The first part has the purpose of placing readers in the Performance Evaluation context. The second part goes deep into the topic, focusing on Performance Evaluation of Knowledge Management.

2.1  Performance management

Organizational Performance Evaluation can be defined as the management process used to build, establish and disseminate knowledge by means of identification, organization, measurement and integration of the aspects of a given context, deemed relevant to measure and manage performance of strategic objectives (ENSSLIN et al., 2014; TADEU DE OLIVEIRA LACERDA et al., 2011).

According to Dutra (2005), having a process to evaluate performance of work structures provides a competitive advantage to managers, as it enables identification and comparison of what was previously planned with what is delivered, showing the level of performance. Knowing what to evaluate and how to evaluate it is critical for valid and legitimate management, allowing managers to make intelligent, transparent and more appropriate decisions, in accordance with their values and contexts (DUTRA et al. 2009; LACERDA et al., 2011).

According to Bititci et al. (2012), the concept of performance evaluation emerged around the 13th century and remained unaltered until the Industrial Revolution. Also, according to this author, as of the 19th century, performance evaluation experienced a process of development marked by phases. The first phase started at the end of the 1880's. This phase was marked by the focus on financial indicators and productivity (GHALAYINI & NOBLE, 1996). The second phase started between 1960 and 1980. Due to the changes in the global market, where organizations started to lose market share to their competitors, the focus on measuring performance has incorporated new performance dimensions, such as quality, time, flexibility and customer satisfaction (BITITCI et al., 2012; GHALAYINI & NOBLE, 1996).

The above mentioned focus shift enabled performance evaluation to be recognized as a multidimensional domain, fostering the development of more integrated approaches, which are unique and balanced in relation to the context (BITITCI et al., 2012). According to Franco-Santos et al. (2012),  contemporary performance evaluation, though giving importance to financial indicators, started to give more attention to indicators related to agility, innovation and interoperability (FRANCO-SANTOS, 2012; VERNADAT et al., 2013).

The changes in the market and in the profiles of business executives have resulted in the development of new performance evaluation systems, as the traditional systems can not lead organizations to achieve their objectives any longer (GHALAYINI & NOBLE, 1996).

2.2 Performance management in knowledge management

The term "Knowledge Management" is defined as a process of creation, acquisition, sharing and the use of knowledge, searching for the expansion of learning capacity and organizational performance (PEE & KANKANHALLI, 2009; ROBINSON et al. 2006). Contrary to the ideas of many individuals, Knowledge Management is not about a simple combination of learning groups or the implementation of an electronic data management system. It is about a management paradigm shift which involves people, culture, information technologies, and organizational structure among others (LEE & KIM, 2001).

The development of a KM strategy, however, allows for an organization to "unlock" different types of knowledge, identify the necessary competencies to plan its future and, through its learning capacity, develop the ability to put sustainability principles into practice (ROBINSON et al. 2006).

Models and benchmarking are used in order to evaluate the acceptance and maturity of any initiative which may impact business processes and their output (HSIEH et al, 2009). Essentially, maturity models describe the development of an entity over time (PEE & KANKANHALLI, 2009). This entity can be anything of interest: a human being, an organizational function, an entrepreneurial initiative, or a technology (HSIEH et al, 2009).

Maturity models are natural applications of the life cycle approach. Every entity develops over time, through different maturity stages, up to reaching its highest maturity level (PEE & KANKANHALLI, 2009; HSIEH et al, 2009). Figure 1 shows the main features and benefits of KM maturity models. 

Figure 1: Features and benefits of KM maturity models

Source: Developed by the authors based on Hsieh et al. (2009).

The benefits from performance evaluation or maturity evaluation of KM, as seen in the study by Hsieh et al. (2009), shown in figure 1, give rise to the conditions for the existence of a sustainable environment for knowledge management and to create competitive conditions of agility, innovation and interoperability dimensions (VERNADAT et al., 2013).

3. Research methodology

The section about the Research Methodology aims to create information for readers on how the research was designed and conducted to achieve its objective and results (LACERDA et. al, 2015).

This section is composed of the methodological framework, description of the intervention instrument called ProKnow-C (Knowledge Development Process – Constructivist) and of data collection procedures.

3.1 – Methodological  Framework

The process of identification and analysis of articles in the bibliographic portfolio took place through the evaluation and interpretation by the authors of this research, qualifying it according to the problem approach, as qualitative research (CRESWELL, 2013).

According to Creswell (2013), for qualitative research, a process named "cross-checking code" (Page 219) must be developed. This process aims for the research to prove reliable and took place through meetings with the researchers, to compare key words individually suggested and, later on, as a group, to analyze and select the key words that best represent the axes of the research.

The accuracy of results was measured by the following validity strategies: (1) "member checking", page 226, by one of the authors of the research, and an expert in Proknow-C, consisting of verifications of the development of each activity performed by the other researchers. These verifications were also complemented by meetings held every week in order to validate the procedure done and legitimize results achieved. (2) "External auditor", page 227, not an author of this research, validated the results after a detailed analysis of the procedure done and of the results achieved.

Regarding technical procedures used, this research is classified as bibliographic as it comprises the identification and analysis of articles published on the selected databases. Regarding data collection, this research used primary and secondary data (YIN, 2003):  primary data in the selection of the bibliographic portfolio, as the researchers´ perceptions and their boundaries are the ones that contribute to the identification of the articles in the BP; secondary data, as the articles in the BP are the ones analyzed. Data collection took place between March and May 2015.

Both data collection and its analysis were guided by the instrument of intervention known as Knowledge Development Process-Constructivist.

3.2 -Knowledge Development Process–Constructivist (ProKnow-C)

This research uses the instrument of intervention known as Knowledge Development Process – Constructivist (ProKnow-C), developed by LabMCDA/UFSC (Laboratório de Avaliação de Desempenho Multicritério em Apoio à Decisão /Universidade Federal de Santa Catarina) (Laboratory of Multicriteria Decision Aid in Performance Evaluation/ Universidade Federal de Santa Catarina) and coordinated by Professor Leonardo Ensslin in 2009.

The main purpose of ProKnow-C is to aid in the process of knowledge development for researchers, in accordance with their research boundaries and interests, from a constructivist point of view (DUTRA et. al, 2015). In order to achieve this purpose, ProKnow-C is used in four different stages: (1) Bibliographic Portfolio Selection; (2) Bibliometrics; (3) Systemic Analysis; and (4) Research Question Objectives.In this research, the two first stages are used: Bibliographic Portfolio SelectionBibliometrics (LACERDA et. al, 2014).

3.3 - Procedures for Data Collection - Bibliographic Portfolio Selection

The stages to compose the BP, as per Proknow-C are: (1) Selection of the Gross Bibliographic Portfolio; (2) Filtering of Gross Bibliographic Portfolio; and (3) Bibliographic Portfolio Representativeness Test. The implementation of these steps is described below.

3.3.1 - Selection of Gross Bibliographic Portfolio

The selection of the articles from the gross portfolio is the first stage to compose the BP. It regards the selection of a set of publications from scientifically relevant databases, and aligned by keywords with the research topic (ENSSLIN et. al, 2014).

The process is initiated by defining the axes of the research and their respective keywords, according to the researchers' perception from their chosen topic. The main axis, which defines the research field of study, refers to performance evaluation and is represented by the following keywords:  "performance", "assessment", "measure", "evaluation" and "maturity". The second axis, which regards the specific topic about which the researchers wish to learn, refers to knowledge management, and is represented by keywords "knowledge management".

When the axes were intersected, a combination of keywords was composed which resulted in 5 search alternatives for each selected database.

After defining the axes of the research, keywords and combinations of alternatives, the following databases were selected: Scopus, Web of Science (or ISI) and EBSCO. These databases were selected as they retrieve more articles based on the keywords used.  

The retrieval of articles from the databases was performed by the search of combinations of the keywords in titles, keywords and abstracts, restricting the publishing period to January 1st, 2000 and March 25th 2015 and to publications of the "article" type, not including books, editorials and book reviews.The result of this search was a set of 8.701 articles named "Preliminary Gross Bibliographic Portfolio." To conclude this stage, an adherence test was done with the keywords, where, through the selection of five articles aligned with the topic and in the Preliminary Gross Bibliographic Portfolio, there was not the need of adding new keywords and new articles to this portfolio, making it the "Gross Bibliographic Portfolio".

It is important to highlight that, initially, the process of article selection was performed using three axes of the research (performance evaluation, knowledge management, healthcare) with their respective keywords. As previously mentioned, the third axis represents the focus of this paper, which is healthcare.  Nevertheless, the use of the three axes resulted in a reduced number of articles, aligned with the research topic, which shows the lack of publications, particularly in healthcare, on performance evaluation of knowledge management. This fact led the researchers to adopt the two first axes of the research, considering that: 1) the research with the two first axes would include publications in healthcare; 2) the identification of a BP on performance evaluation of knowledge management, overall, could support the development of future studies about healthcare, given the reduced number of identified publications.

3.3.2 - Filtering of Gross Portfolio of Articles

The filtering process took place with the aid of the Endnote software where, through a tool that eliminates duplicated articles, 1.798 articles were rejected, resulting in 6.903 unrepeated articles.

As the next stage, reading of 6.903 titles of the unrepeated articles took place, searching for publications aligned with the topic research. At this stage, 6.483 were rejected.

420 unrepeated articles and aligned by the reading of the titles were subject to the tests in Figure 2: Filtering as for Scientific Recognition.

Figure 2: Filtering as for Scientific Recognition

Source: Tasca et al.(2010).

The tests in figure 2 evaluate the publications regarding pre-set criteria: number of citations, year when the article was published, and alignment with the topic after reading of their abstracts, among others. At the end of the tests, 44 articles were obtained. For these articles, their availability on the databases of the Universidade de Santa Catarina (Unisul) and of the Coordination for the Improvement of Higher Education Personnel (Capes) was checked as well as their reading later in order to eliminate publications which were not aligned with the research topic. At this stage, 7 articles were not available and 18 of them were not aligned with the research topic. The 19 resulting articles composed the "Preliminary Bibliographic Portfolio".

3.3.3 - Bibliographic Portfolio Representativeness Test

With the Preliminary Bibliographic Portfolio composed, its bibliographic references were listed, with a total of 906 references. These references were filtered as to their timeline – from January 1st, 2000 to March 3rd, 2015 – and as to "type of file" – article -, resulting in 419 accepted articles. Of these 419 articles, 7 of them were identified as being part of the articles in the Preliminary Bibliographic Portfolio and excluded from this stage.

The 412 articles were imported to the software Endnote, subject to filtering for the elimination of duplicate articles and analysis of titles aligned with the research topic. This process resulted in the elimination of 389 articles, and 23 of them were kept. Of these 23 articles, 4 of them were rejected due to the fact they were not sufficiently scientifically relevant (the number of citations was lower than the established level of significance) and 16 of them were not entirely aligned with the research topic. 

The 3 remaining articles were added to the 19 articles in the Preliminary Bibliographic Portfolio, which resulted in the desired "Bibliographic Portfolio", composed of 22 articles. Of these 22 articles, only 3 of them refer to the performance evaluation of knowledge management in healthcare.

4. Results found       

The Results Found section is composed of the presentation of Bibliometrics. Seeking a higher level of clarity, this section was subdivided into 4 topics: (1) Prominent periodicals in the BP and in its references; (2) Prominent articles in the BP and in its references; (3) Prominent authors in the BP and in its references; and (4) Impact Factor of The Journals Found.

4.1 – Prominent periodicals in the BP and in its references

The first aspect analyzed was the receptivity of the journals regarding the research topic, which also contributes to the identification of journals which are open to publish research on the topic. The 7 most prominent journals, according to the result of intersecting the journals in the BP and its references are shown in Figure 3: Prominent Journals in Articles and References of the Bibliographic Portfolio.

Figure 3: Prominent Journals in Articles and References of the Bibliographic Portfolio

Source: Authors Research data (2015).

Journal of Knowledge Management and Expert Systems with Applications are the journals in a position of prominence, both in the Bibliographic Portfolio and in its references.

Journal of Knowledge Management is a journal that is part of Emerald Group Publishing. This journal is dedicated to exchanging academic research and practical information on the aspects of knowledge management in organizations. The focus of the journal is on the identification of Knowledge Management innovative strategies and in the application of theoretical concepts to real world situations.

Expert Systems with Applications is an international journal whose focus in on exchanging information related to expert and intelligent systems applied in industries, governments and universities worldwide. The thrust of the journal is (1) to publish papers dealing with the design, development, testing, implementation, and/or management of expert and intelligent systems, and also (2) to provide practical guidelines in the development and management of these systems.

4.2 - Prominent articles in the BP and in its references

The second aspect analyzed was the one that shows prominent articles and authors in the BP and in its references. Figure 4 - Prominent articles and authors of the Bibliographic Portfolio - shows that no selected publications in this fraction of the literature conform to "prominent article written by a prominent author". The fraction made it possible to identify 1 prominent article and 2 articles written by prominent authors.

Figure 4: Prominent articles and authors of the Bibliographic Portfolio

Source: Authors Research data (2015).

The article entitled "KMPI: measuring knowledge management performance" by Chang, L. K. et al, 2005 stands out as the most cited article in the BP on Google Scholar, with a total of 388 citations. The articles entitled "STEPS: A knowledge management maturity roadmap for corporate sustainability" and "A stage model of organizational knowledge management: A latent content analysis" stand out as articles written by prominent authors in the BP references, with 7 and 6 citations, respectively. The lack of prominent articles in the BP with prominent authors in its references, Figure 4: Prominent articles and authors of the Bibliographic Portfolio, shows that research on the topic is still at an early stage.

4.3 - Prominent authors in the BP and in its references

This section presents the prominent authors in the Bibliographic Portfolio (BP), in its references and in the Gross Bibliographic Portfolio of this fraction of the literature. It was identified that the BP was written by 48 authors, of whom none of them have more than one selected article in the BP. This proves that no author has dedicated to the Performance Evaluation of Knowledge Management. Next, the same information was analyzed in the references cited by the articles in the BP and in the Gross Bibliographic Portfolio , regarded as aligned with the topic in question. Figure 5 - Prominent authors in articles of the Bibliographic Portfolio and in its References and Gross Bibliographic Portfolio - shows that authors in the BP were cited by many other authors in the BP and in the Gross Bibliographic Portfolio, showing their presence in the field of study.

Figure 5: Prominent authors in articles of the Bibliographic Portfolio and in its References and Gross Bibliographic Portfolio

Source: Authors Research data (2015).

Figure 5 - Prominent authors in the articles of Bibliographic Portfolio and in its References and Gross Bibliographic Portfolio - also shows that the Gross Bibliographic Portfolio is similar to the Bibliographic Portfolio, though it does not take its proportionalities into account.  This indicates that the authors in the BP are not limited to their area of work, only to this fraction of this topic, but also to related topics. It is possible to note that some authors do not have articles in the Gross Bibliographic Portfolio. These authors were added to the author database through the articles selected by the representativeness test done before the selection of the Gross Bibliographic Portfolio. Another interesting fact is that, although the authors are cited once in the BP, they are cited more often in the references, which suggests they are relevant to the related area of work.

Chimay J. Anumba is the most cited author in the BP references, with 9 citations. Chimay J. Anumba holds a Ph.D in Civil Engineering. His research interests are in the fields of Artificial Intelligence/Knowledge-Based Systems/Knowledge Management, Advanced Engineering Informatics, Concurrent Engineering in Construction, Distributed Collaboration and e-Business and Construction. He is a professor of Architectural Engineering at the Pennsylvania State University/USA, and by analyzing his publications, it is possible to see that he prefers to work with other authors. He has a total of 6.680 citations on Goggle Scholar and 137 publications with more than 10 citations.

The second place, according to the number of citations in the BP references is held by Chinho Li, with 8 citations. Lin is the Vice President and a Professor of the National Cheng Kung University in Taiwan. His research interests are in the fields of electric power engineering, power system economic dispatch, power electronics, new energy system, avionics systems, flight control, magnetic suspension system, and recent mobile communications. By analyzing his publications, it is possible to see that he prefers to work with other authors. Lin has already published 120 articles in journals and in more than 300 articles for congresses. His profile is not on Google Scholar.

The third most cited author in the BP references is Herbert Robinson. Robinson is the Regional Adviser and Head of the Training Division at United Nations African Institute for Economic Development & Planning/Economic Commission for Africa/UK. His industrial experience includes major construction project in Arup, in the United Kingdom, and in projects funded by the World Bank in Gambia.  His current research focus is on knowledge management and improvement of business performance and on infrastructure projects funded by the private sector. By analyzing his publications, it is possible to see that he prefers to work with other authors. His profile is not on Google Scholar.

4.4 - Impact factor of The Journals Found

The purpose of this section is to present the impact factor of the BP journals, based on Isi and Scopus databases.  Figure 6 - Impact factor of the journals of the PB – shows a list of impact factors found on these two databases.

Figure 6: Impact factor of the journals of the PB

Source: Authors Research data (2015).

Figure 6 shows that Knowledge-based Systems, Knowledge and Information Systems and Expert Systems with Applications are the journals with the highest Impact Factor. The other 15 journals analyzed show a lower impact factor.

5. Conclusions                    

Based on this paper´s research question, How to select a limited set of scientifically relevant articles aligned with performance evaluation of knowledge management in healthcare, and based on this bibliographic portfolio, conduct bibliometrics on the following parameters: journals, articles and authors?, it was established that the purpose of selecting international scientific publications as of the year 2000, which deal with a fraction of the literature on Performance Evaluation of Knowledge Management, making it possible to present, from this perspective of the topic, their most present journals, articles and authors.

The research question was answered by means of the instrument of intervention called Knowledge Development Process – Constructivist (ProKnow-C) for bibliographic portfolio (BP) selection and bibliometrics, as it has as a structured process susceptible to replication and an aligned constructivist point of view; therefore, aligned with the purpose of the research.

The selection of the Bibliographic Portfolio was made based on the Scopus, Web of Science (ISI) and EBSCO databases, and on a timeline, from the year 2000 to 2015. Through the find function - "performance" or "assessment" or "measure*" or "evaluation" or "maturity" and "knowledge management" - 8701 articles were found. Of these articles, after the filtering processes, 8.679 articles were rejected, resulting in a BP composed of 22 articles. The analysis of these articles that compose the BP could prove that:

1. Journals of Knowledge Management and Expert Systems with Applications are journals in a position of prominence, both in the Bibliographic Portfolio (BP) and in its references. These journals are dedicated to exchanging academic research and practical information on the aspects of knowledge management in organizations and specialist systems.
2. There were no publications, selected in this fraction of the literature, which conformed to "prominent article written by a prominent author". This can be explained due to fact this topic is relatively new, with no author/journal dedicated to the topic that could be cited.  The fraction of the literature made it possible to identify 1 prominent article and 2 articles written by prominent authors.
3. Chimay J. Anumba is the most cited author in the BP references, with 9 citations. Chimay J. Anumba holds a Ph.D in Civil Engineering. His research interests are in the fields of Artificial Intelligence/Knowledge-Based Systems and Knowledge Management, with no research dedicated the Evaluation Performance of Knowledge Management. This is also explained by the fact the topic is new, with no new authors with a tradition in it.

The article entitled "KMPI: measuring knowledge management performance" by Chang, L. K. et al, 2005 stands out as the most cited article in the BP on Google Scholar, with a total of 388 citations. This means that the research topic is new, resulting in a lack of authors and/or journals with systematic publications on the topic.

The research process with the use of Proknow-C also allowed the authors to identify the bibliographic portfolio, representative of a fraction of the topic as perceived by them, presenting the journals that publish the topic the most cited articles, their authors and their scientific recognition, their research places and whether they work by themselves or with other authors, allowing a better understanding of how and where the topic is being researched.    

It was noticed that the most cited authors in the BP references prefer to work with other authors. C. J. Anumba works in the United States mainly in Civil Construction; Chinho Lin works in Taiwan mainly in Electric Engineering and Herbert Robinson works in the United Kingdom in knowledge management and improvement of business performance and in infrastructure projects funded by the private sector.   

These facts show a small combination of interests of the prominent authors, which is justified by the short time in which this topic has been explored scientifically; however, uniformity of the approaches to its study is  needed. This knowledge will help in the continuity of future research of this line of research as well as building the base to argue about selected theoretical affiliations.

This research has proved that the field of study related to Hospital Knowledge Management informed by Performance Evaluation represents a fertile field to be explored due to the fact there are no authors dedicated to the study of this topic and prominent in it. This is explained by the fact this is a relatively new topic and not explicitly added to the editorial purposes of the journals.

Among the limitations found, the following ones are highlighted: (1) in the choice of databases, the ones focused on the topic of management were prioritized; (2) the composition of the Bibliographic Portfolio by scientific publications available on databases accessed through the Universidade de Santa Catarina (Unisul) and through the journal website of the Coordination for the Improvement of Higher Education Personnel  (Capes); the latter is available for online search in universities in Brazil; (3) the limitation by time of articles published after the year 2000; and (4) the operationalization of the two first stages of the ProKnow-C.

For future research, these are suggested: (1) continuity of this research, with the development of stages of "systemic analysis" and "identification of research opportunities" by means of Proknow-C, based on setting a theoretical affiliation of the researchers; (2) the development of more studies combining "Performance Evaluation", "Knowledge Management", and "Healthcare" topics, as it is a field of study yet to be more fully explored.

References

Bititci, U., Garengo, P., Dörfler, V., & Nudurupati, S. (2012). Performance measurement: Challenges for tomorrow*. International Journal of Management Reviews, 14(3), 305-327.

Colauto, R. D., & Beuren, I. M. (2003). Proposta para avaliação da gestão do conhecimento em entidade filantrópica: o caso de uma organização hospitalar. Revista de Administração Contemporânea, 7(4), 163-185.

Creswell, J. W. (2013). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.

Dutra, A. (2005). Metodologias para avaliar o desempenho organizacional: revisão e proposta de uma abordagem multicritério. Revista Contemporânea de Contabilidade, 2(3), 25-56.

Dutra, A., Ensslin, L., Ensslin, S. R., Lima, M. V. A., & Lopes, A. L. M. (2009). Inovação no Processo de Avaliação do Desempenho Organizacional: O Uso da Dimensão Integrativa DOI: 10.5585/rai. v5i2. 244. RAI: revista de administração e inovação, 5(2), 150-163.

Dutra, A., Ripoll-Feliu, V. M., Fillol, A. G., Ensslin, S. R., & Ensslin, L. (2015). The construction of knowledge from the scientific literature about the theme seaport performance evaluation. International Journal of Productivity and Performance Management, 64(2).

Ensslin, S. R., Ensslin, L., de Oliveira Lacerda, R. T., & de Souza, V. H. A. (2014). Disclosure of the State of the Art of Performance Evaluation Applied to Project Management. American Journal of Industrial and Business Management, 4(11), 677.

Franco-Santos, M., Lucianetti, L., & Bourne, M. (2012). Contemporary performance measurement systems: A review of their consequences and a framework for research. Management Accounting Research, 23(2), 79-119.

Ghalayini, A. M., & Noble, J. S. (1996). The changing basis of performance measurement. International Journal of Operations & Production Management, 16(8), 63-80.

Hsieh, P. J., Lin, B., & Lin, C. (2009). The construction and application of knowledge navigator model (KNM™): An evaluation of knowledge management maturity. Expert Systems with Applications, 36(2), 4087-4100.

Lacerda, R. T., Ensslin, L., & Ensslin, S. R. (2015). Research Methods and success Meaning in Project Management. Designs, Methods and Practices for Research of Project Management, Page 85 - 93. Edited by Beverly Pasian.

Lacerda, R. T. D. O., Ensslin, L., & Ensslin, S. R. (2014). Research opportunities in strategic management field: a performance measurement approach. International Journal of Business Performance Management, 15(2), 158-174.

Lee, J. H., & Kim, Y. G. (2001). A stage model of organizational knowledge management: a latent content analysis. Expert Systems with Applications, 20(4), 299-311.

Lotti Oliva, F. (2014). Knowledge management barriers, practices and maturity model. Journal of Knowledge Management, 18(6), 1053-1074.

Pee, L. G., & Kankanhalli, A. (2009). A model of organisational knowledge management maturity based on people, process, and technology. Journal of Information & Knowledge Management, 8(02), 79-99.

Robinson, H. S., Anumba, C. J., Carrillo, P. M., & Al-Ghassani, A. M. (2006). STEPS: a knowledge management maturity roadmap for corporate sustainability. Business Process Management Journal, 12(6), 793-808.

Tadeu de Oliveira Lacerda, R., Ensslin, L., & Rolim Ensslin, S. (2011). A performance measurement framework in portfolio management: A constructivist case. Management Decision, 49(4), 648-668.

Tasca, J. E.; Ensslin, L.; Ensslin, S. R. & Alves, M. B. M., (2010). An approach for selecting a theoretical framework for the evaluation of training programs. Journal of European Industrial Training, v. 34, n. 7, p. 631-655.

Vernadat, F., Shah, L., Etienne, A., & Siadat, A. (2013). VR-PMS: a new approach for performance measurement and management of industrial systems. International Journal of Production Research, 51(23-24), 7420-7438

Wen, Y. F. (2009). An effectiveness measurement model for knowledge management. Knowledge-based systems, 22(5), 363-367.

Yin, R. K. (2003). Case study research: design and methods. 3th Edition. Thousand Oaks: Sage Publications.

1. PhD from the University of Lancaster - England (2000), Doctor in Industrial and Systems Engineering at the University of Southern California (1974), master's degree in Production Engineering from the Federal University of Santa Catarina - UFSC (1970), Degree in Mechanical Engineering from the Federal University of Rio Grande do Sul – UFRGS (1968)
2. Doctor in Management from the University of São Paulo - USP (2008), Master in Business Administration from the Federal University of Santa Catarina - UFSC (2002), degree in Computer Science from the Federal University of Santa Catarina - UFSC (1996).
3. PhD from the University of Valencia - Spain (2014), Doctor in Production Engineering from the Federal University of Santa Catarina - UFSC (2002), master's degree in Production Engineering from the Federal University of Santa Catarina - UFSC (1995), degree in Accounting Sciences from the Catholic University of Pelotas (1991);

4. PhD from the University of Valencia - Spain (2004), Doctor in Production Engineering from the Federal University of Santa Catarina - UFSC (2003), master's degree in Production Engineering from the Federal University of Santa Catarina - UFSC (1998), degree in Business Administration from the Federal University of Santa Catarina - UFSC (1988);

5. Specialist IT Project Management from the University of Southern Santa Catarina - UNISUL (2013), degree in Information Systems from the Federal University of Santa Catarina - UFSC (2007). email: xandefm@gmail.com