Zeszyty Naukowe 24(96) 25
Scientific Journals
Zeszyty Naukowe
Maritime University of Szczecin
Akademia Morska w Szczecinie
2010, 24(96) pp. 25–27 2010, 24(96) s. 25–27
The questionnaire in the conditions of uncertainty
Ankieta w warunkach niepewności
Jacek Mazurkiewicz
Silesian University of Technology, Faculty of Organisation and Management Institute of Production Engineering
Politechnika Śląska w Gliwicach, Wydział Organizacji i Zarządzania, Instytut Inżynierii Produkcji 41-800 Zabrze, ul. Roosevelta 26, e-mail: jacekma1@tlen.pl
Key words: poll survey statistic test, credibility, indirect questions Abstract
The study refers to questionnaires in which on the basis of scoring of each question, it can be concluded about a formulated research hypothesis. The following paper examines ways of developing, instituting and analyzing surveys in order to accept or reject stated hypothesis. In particular the focus is on measuring credibility of answers provided by participants in the survey. Each answer is scored along two criteria: person’s credibility and significance of the question. The final score is a weighted average of those two variables. This method is particularly useful in surveys when anonymity of participants is not sufficient to guarantee a credible response.
Słowa kluczowe: ankieta, testowanie, wiarygodność, pytania pośrednie Abstrakt
Opracowanie dotyczy ankiet, z których w oparciu o punktacje każdego z pytań można wnioskować o posta-wionej hipotezie badawczej. Artykuł opisuje sposoby rozwoju, wszczynania i analizy badań, w celu przyjęcia lub odrzucenia postawionej hipotezy. W szczególności koncentruje się na określeniu wiarygodności odpo-wiedzi udzielonych przez uczestników badania. Każda odpowiedź jest oceniana według dwóch kryteriów: wiarygodności osoby i znaczenia pytania. Wynik końcowy jest średnią ważoną tych dwóch zmiennych. Metoda ta jest szczególnie przydatna w badaniach, w których anonimowość uczestników nie jest wystarcza-jąca, by zagwarantować wiarygodną odpowiedź.
Introduction
It is expected from a questionnaire to be con-ducted on a reliable research group (representative) and the obtained answers must be credible. Only two conditions however, in most of the cases they are not even partly fulfilled. It requires a choice of representative group from all population and then, studied characteristics of population should be as-sessed on the basis of statistics methods. It is more difficult to obtain credible answers, especially in case of situations when a studied research group’s political attitudes suppress respondents’ impression (of opinion) and what’s more, there would appear a lack of sense of total anonymity. Furthermore, we should be aware that questionnaires are only true for a given place and moment, and not beyond them.
The questionnaire research is used for different kinds of work where the application of one type of research is not enough or it is hindered. This research is often thought to be very easy and an effective research tool – unfortunately, the situation is contradictory. The unquestioning use of ques-tions, their configuraques-tions, choice of respondents and lack of statistical study have here a significant meaning as it decides about the value of research as well as about its results. The questionnaire, if it is possible, should be only one of the research methods of analysed problem and its results should be compared with the results of other method.
The aim of this work is to analyse the elements of questionnaire as we think that such an analysis can be useful for those who start this kind of research.
Jacek Mazurkiewicz
26 Scientific Journals 24(96)
The basic elements of a questionnaire
The aim of questionnaire research
The aim of questionnaire is the beginning of its creating as precise defining allows to control the work over the whole questionnaire. This aim is connected with formulating a hypothesis that is a general presumption, which is assessed by means of questionnaire survey. At this stage, conditions which are either sufficient for acceptation or rejec-tion of a formulated hypothesis are made.
It can be assumed for example, that employees’ knowledge of quality management system, which is used, should gain a level, which is perceived as satisfactory if for example: 2/3 of correct answers are accepted.
The research can concern for instance: the as-sessment of documents flow, if time of documents flow or troublesomeness of their study is not bigger that the troublesomeness of providing a service then, the results can be considered to be satisfac-tory.
Naturally, defining the aim as well as formu-lating a hypothesis depends on the scope and aim of research for which a questionnaire is used.
The technique of conducting a questionnaire
The choice of technique is essential due to the possibilities of gaining credible results or their checking. Generally, a questionnaire survey is con-ducted indirectly, by means of giving a respondent a questionnaire or directly, by means of asking him questions. The differences and results are funda-mental as in the direct survey, more information which is connected with studied aim of a question-naire, with credibility of answers and situations in which a respondent was obtained. Thus, the credi-bility of answers can be assessed and besides this, there is a random choice which is very important in this type of research. The randomness in this re-search mainly depends on our planning the survey.
In case of sending questionnaires to respondents, firstly there is no control over them filling up and secondly, only few people who fill questionnaires up and send them back so a very significant ele-ment of randomness does not occur in this case. Most often, no more than 10% of respondents send back filled up questionnaires. In this case, the basic question is what group in this survey constitutes this 10% and how representative this group is for our research. Certainly, it is not a random group.
The questions testing a hypothesis (first layer)
Couple of questions which unfortunately, can’t be always asked directly to a respondent constitute the main questions whose answers will allow to
assess if formulated hypothesis is true or false. It is expected that a respondent’s answer will not be probably true. It can’t assumed that if a dealer is asked about a quality of his products, he will not answer that they are not so good or a plenipoten-tiary of quality will not talk negatively about a sys-tem which he has implemented or an employee will not tell that he robs his company. There are a lot of such situations and in such cases the layer-framework is accepted. It is about creating ques-tions separately for respondents which will be a basis for our answers for separately prepared main questions which are the main aim of questin-narie rearch. A good example is a question for an employee of company’s car service: How often do this company’s cars break down? The answer would be “very seldom” while a question: how effective are the staff and how many cars are repaired every day? allow to assess a real frequency of breakdowns of cars in a given company. So, depending on the aim of a questionnaire, real direc-tions of survey will be hidden or not. At this stage of constructing the questions of questionnaire, a decision about value of a questionnaire is taken.
The questions for respondents (second layer)
Defining questions for respondents refers to using the layer framework of a questionnaire where it is expected to get answers to the first layer. Creating a set of questions depends to a great extend on empathy of a person creating a question-naire but it also depends on knowledge of studied issue, environment and experience. There are many techniques of creating this type of questions which are described in literature. However, the analysis of questions used in a questionnaire very often provides better effect.
Defining scoring, scale
The need to use a scale as well as scoring of questions and answers is a consequence of the fact of surveying a chosen group, not the whole popula-tion. It is a great mistake to directly refer results of a subgroup to the whole population. In order to prevent it, it is advised to use statistics methods which are connected with prioritizing and itemizing the results. Testing hypothesis also expects similar requirements.
Scoring
There are no basis to claim that each question for the purpose of a questionnaire is equally impor-tant thus, each and every should have a separate scoring. The choice of scoring is ,in many cases an arbitrary decision of questionnaire’s creator but in
The questionnaire in the conditions of uncertainty
Zeszyty Naukowe 24(96) 27
many cases it should be justified. It is simpler to accept the same scoring for all answers but at the end, it should be corrected by attributing values to each question. What seems to be more convenient as it gives an identical end effect.
Scale
The scale which is mostly used is a 3-, 5- or 7- -point scale with a possibility of crossing out the answer if a respondent is not familiar with a topic or does not want to answer, which happens quite often. The central point responds to a neutral assessment and the rest is chosen is such a way that a size of negative and positive assessment was the same.
The technique of credibility of respondents
Despite earlier trials to prevent the occurrence of tendentious answers, the credibility of separate respondents should be done. There are many tech-niques of assessment, starting from subjective, in case of direct contact, by means of questions about facts of a given company which a creator of a ques-tionnaire know e.g.: a real rate of flaws and com-plaints, to events concerned directly with respon-dents which in many case is similar to the previous one. The test questions are also used to check a respondent’s knowledge of topic which helps in assessment of his competence to give reliable answers referring to a scope of research. These kinds of questions should not be prominent; they are usually hidden among other questions so as not to make respondents uneasy.
The analysis of questionnaire questions
The analysis refers both to a formal and factual side.
Attributing values to credibility
As a result of credibility of assessment of each respondent, values are attributed to them, that is, a degree of credibility usually from the whole cer-tainty of responding value 1 up to the value 0 which is equivalent with rejecting an answer. Apart from a subjective assessment of credibility estimated during a contact with a respondent, the analysis of credibility is done. It is based on a study of correla-tions between test quescorrela-tions and assumpcorrela-tions on the basis of which the questions were created. The values of credibility are taken into account during further estimations. It refers to average values, va-riance, standard deviation or other statistical sizes. The assessment of distribution of answers It is claimed that the distribution of answers should be similar to normal one. Checking the
normality of distribution is done by means of graphs or histograms. Seldom, but more precise it is conducted by means of Chi-square test or Kolgo-morov’s test.
Trust range of studied questionnaire’s parameters
In order to define what evaluation and with what accuracy it was defined, a trust range of answers is built. The rules of its building are identical to statis-tics descriptions. While counting the average scor-ing values of studied answers, the significance of credibility is taken into account for all respondents and a similar procedure is used when estimating the standard deviation. The level of freedom depends on amount of trials, that’s on respondents. In order to define a trust range, the level of gravity – alfa should be accepted, that is a level of acceptable error which is defined by a probability of occur-rence of average from general population beyond defined range for a studied parameter. Most often, alfa is = 0.05. When knowing these values, the trust range, which is accepted as an accuracy range for a studied parameter, is defined.
Statistical verification of hypothesis
Testing the initially accepted statistical hypothe-sis – so called a zero hypothehypothe-sis is conducted after accepting a level of gravity, that is a boarder value of probability which when being exceeded, is not accepted. Rejecting the zero hypothesis is mostly tantamount to accepting a contradictory hypothesis. The level of gravity is, as described earlier, under-stood, in this case as a minimal probability when a zero hypothesis is accepted. If a probability of described event is smaller, the zero hypothesis is rejected. Accepting the zero hypothesis does not mean that it is real. It only means that there is no basis to reject it.
Verifying aims of questionnaire (are they in accordance with prediction?)
Research, which was conducted, is a special kind of studies and is based on close questions which means that we are so familiar with an issue that all possible, interesting answers were predicted and pointed out and a hypothesis which was formu-lated an as we think is correct and is supported with research. In order to check if it is real like this, we can look through the questionnaire’s results and compare effects with the defined aim. If not, it means that we do not have enough knowledge of what we study.
Recenzent: prof. dr hab. inż. Krzysztof Chwesiuk Akademia Morska w Szczecinie
