LIU Lei, WU Yu-feng, LI Xiao-jun. Current Practice in Measuring the Quality of Conceptual Models:Challenges and Research[J]. International Journal of Plant Engineering and Management, 2012, 17(3): 136-144

Current Practice in Measuring the Quality of Conceptual Models:Challenges and Research
LIU Lei1
1. The Second Artillery Engineering University, Xi'an 710025, P.R.China;
2. School of Management, Xi'an Jiaotong University, Xi'an 710049, P.R.China
How to measure the quality of conceptual models is an important issue in the IS field and related research. This paper conducts a review of research in measuring conceptual model quality and identifies the major theoretical and practical issues that need to be addressed in future studies. We review current classification frameworks for conceptual model quality and practice of measuring conceptual model quality. Based on the review, challenges for studies of measuring the quality of conceptual models are proposed and these challenges are also research points which should be strengthened in future studies.
Key words:    conceptual model quality    measurement    syntactic quality    semantic quality    pragmatic quality   
Received: 2012-07-06     Revised:
Funds: This paper is supported by the Programs Foundation of Military Theoretical Research under Grant No. 2011C026
Corresponding author:     Email:
Author description:
PDF(781KB) Free

[1] Siau K,Informational and computational equiva-lence in comparing information modeling meth-ods.Journal of Database Management,15(1):73~86,2004
[2] Yang F Q,Thinking on the development of soft-ware engineering technology.Journal of Soft-ware,16(1):1~7,2005 (In Chinese)
[3] Wand Y,Weber R A,Research commentary: in-formation systems and conceptual modeling-a re-search agenda.Information Systems Research,13(4):363~376,2002
[4] Fan Y,Li W M,Research on the formal descrip-tion language for military conceptual modeling.Fire Control and Command Control,31(6):19~22,2006 (In Chinese)
[5] Enders A,Rombach H D,A handbook of soft-ware and systems engineering: empirical observa tions.Laws and Theories,Addison-Wesley,Reading,MA,USA,2003
[6] Lauesen S,Vinter O,Preventing requirement defects.In:Proceedings of the 6th Internation-al Workshop on Requirements Engineering:Foundation for Software Quality,REFSQ 2000,Stockholm,Sweden,2000
[7] Boehm B W,Software engineering economics.Prentice-Hall,Englewood Cliffs,USA,767,1981
[8] Daniel L,Moody,Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions.Data & Knowledge Engineering,55:243~276,2005
[9] ISO,ISO Standard 9000-2000: Quality man-agement systems: fundamentals and vocabula-ry.International Standards Organization(ISO),2000
[10] Sun F,Quality evaluation of conceptual models from users perspectives.Application of Computer Systems,12:166~170,2008(In Chinese)
[11] Bansiya C,Davis A,Hierarchical model for object-oriented design quality assessment.IEEE Transactions on Software Engineering,28(1):4~17,2002
[12] ISO/IEC,ISO/IEC Standard 9126: Software Product Quality,International Standards Or-ganization (ISO).International Electro-tech-nical Commission (IEC),2001
[13] Lindland O I,Sindre G A,Slvberg,Under-standing quality in conceptual modeling.IEEE Software,11(2):42~49,1994
[14] Moody D L,Shanks G,Improving the quality of data models: empirical validation of a qual-ity management framework.Information Sys-tems,28(6):619~650,2003
[15] Moody D L,et al,Evaluating the quality of process models: empirical testing of a quality framework.Lecture Notes in Computer Sci-ence,2503:380~396,2002
[16] Moody D L,Sindre G,Brasethvik T,A Slv-berg,Evaluating the quality of information models: empirical testing of a conceptual mod-el quality framework.In: Proceedings of the 25th International Conference on Software En-gineering,IEEE Computer Society,2003
[17] Kesh S,Evaluating the quality of entity rela-tionship models.Information and Software Technology,37(12):681~689,1995
[18] Schütte R,Rotthowe T,The guidelines of modeling-an approach to enhance the quality in information models.Lecture Notes in Computer Science,1507:240~254,1998
[19] Poels G,Nelson J,Genero M,Piattini M,Quality in conceptual modeling-new research directions.International Workshop on Con-ceptual Modeling Quality,IWCMQ' 02,Springer-Verlag,Tampere,Finland,2002
[20] Krogstie J,A Slvberg,Information systems engineering-conceptual modeling in a quality perspective.Kompendiumforlaget,Trondheim,Norway,2003
[21] Siau K,Tan X,Improving the quality of con-ceptual modeling using cognitive mapping tech-niques.Data & Knowledge Engineering,55:343~36,2005
[22] Seddon P B,A re-specification and extension of the delone and mcLean model of is success.Information Systems Research,8(3):240~253,1997
[23] Maes A,Geert Poels,Evaluating quality of conceptual modeling scripts based on user per-ceptions.Data & Knowledge Engineering,63:701~724,2007
[24] Krogstie J,Sindre G,Jrgensen H,Process models representing knowledge for action: a re-vised quality framework.European Journal of Information Systems,15:91~102,2006
[25] Mohagheghi P,Aagedal J,Evaluating quality in model-driven engineering.29th International Conference on Software Engineering Workshops(ICSEW'07),IEEE Computer Society Wash-ington,DC,USA.2007
[26] Mohagheghi P,Dehlen V,Neple T,Defini-tions and approaches to model quality in mod-el-based software development-a review of liter-ature.Information and Software Technology,51:1646~1669,2009
[27] Nelson H J,Monarchi D E,Ensuring the qual-ity of conceptual representations.Software Qual-ity Journal,15(2):213~233,2007
[28] Mohagheghi P,Dehlen V.Developing a qual-ity framework for model-driven engineering.H.Giese ( Ed.): MoDELS 2007 Work-shops,LNCS 5002,275~286,2008
[29] Bolloju N,Felix S K,Leung.Assisting novice analysts in developing quality concept.Com-munications of the ACM,49(7):108~112,2006
[30] Krogstie J,Lindland O I,Sindre G,Defining quality aspects for conceptual models.In: Pro-ceedings of the 3rd IFIP 8.1 Working Confer-ence on Information Systems,Marburg,Ger-many,216~231,1995
[31] Dunn C L,Grabski S V,Perceived semantic expressiveness of accounting systems and task accuracy effects.International Journal of Ac-counting Information Systems,1 (2):79~87,2000
[32] Dunn C L,Grabski S V,An investigation of localization as an element of cognitive fit inaccounting model representations.Decision Science,32(1):55~94,2001
[33] Halpin T A,Comparing meta-models for ER,ORM and UML data models.Advanced Top-ics in Database Research,3:23~44,2004
[34] Gemino A,Wand Y,Foundations for empiri-cal comparisons of conceptual modeling tech-niques.In: Proceedings of the 2nd Annual Symposium on Research in Systems Analysis and Design,Miami,FL,2003
[35] Wand Y,Weber R,On the ontological ex-pressiveness of information systems analysis and design grammars.Journal of Information Systems,3:217~237,1993
[36] Wand Y,Weber R,On the deep structure of information systems.Information Systems Journal,5:203~223,1995
[37] Jan M,Jan R.Extending the discussion of model quality: why clarity and completeness may not always be enough.In Pernici,Bar-bara and Gulla,Jon Atle,Eds.Proceedings The 19th International Conference on Ad-vanced Information Systems Engineering(CAiSE'07),Trondheim,Norway,2007
[38] Kühne T,Matters of (meta-) modeling.Soft-ware and Systems Modeling,5:369~385,2006
[39] Mending J,hajo A,Reijers,Jorge Cardoso,What makes process models understandable.G.Alonso,P.Dadam,M.Rosemenn(Eds.): BPM 2007,LNCS 4714,48~
[40] Parsons J,Cole L,What do the pictures mean? guidelines for the experimental evalu-ation of representation fidelity in diagram-matical conceptual modeling techniques.Da-ta & Knowledge Engineering,55(3):327~342,2005
[41] Bodart F,Patel A,Sim M,Weber R,Should optional properties be used in conceptual mod-elling? a theory and three empirical tests.In-formation Systems Research,12(4):384~405,2001
[42] Kim Y G,March S T,Comparing data mod-eling formalisms.Communications of the ACM,38(6):103~115,1995
[43] Siau K,Wand Y,Benbasat I,The relative importance of structural constraints and sur-face semantics in information modeling.Infor-mation Systems,22(2/3):155~170,1997
[44] Dunn C L,Gerard G J,Auditor efficiency and effectiveness with diagrammatic and linguistic conceptual model representations.International Journal of Accounting Information Systems,2(4):223~248,2001
[45] Burton-Jones A,Meso P N,Conceptualizing systems for understanding: an empirical test of decomposition principles in object-oriented analysis.Information Systems Research,17:38~60,2006
[46] Burton-Jones A,Weber R,Properties do not have properties: Investigating A Questionable Conceptual Modeling Practice.In: Proceed-ings of the 2nd Annual Symposium on Re-search in Systems Analysis and Design,Mi-ami,FL,2003
[47] Gemino A,Wand Y,Complexity and clarity in conceptual modeling: comparison of man-datory and optional properties.Data & Knowl-edge Engineering,55(3):301~328,2005
[48] Muckler F A,Seven S A,Selecting perform-ance measures: objective versus subjective meas-urement.Human Factors,34(4):441~455,1992
[49] Annett J,Subjective rating scales science or art? Ergonomics,45(14):966~987,2002
[50] Deming W E,Out of the crisis.MIT center for advanced engineering.Cambridge,MA,1986
[51] Evans J R,Lindsay W M,The management and control of quality.6th ed.,South-West-ern College Publishing (Thomson Learning), Cincinnati,USA,2004,848
[52] Neuman W L,Social research methods-quali-tative and quantitative approaches.4th ed.,Allyn and Bacon,Needham Heights,MA,USA,2000