• Nie Znaleziono Wyników

Ewaluacja w edukacji : teoria i praktyka

N/A
N/A
Protected

Academic year: 2022

Share "Ewaluacja w edukacji : teoria i praktyka"

Copied!
132
0
0

Pełen tekst

(1)
(2)
(3)

Ewaluacja w edukacji –

teoria i praktyka

(4)
(5)

Ewaluacja w edukacji – teoria i praktyka

pod redakcją

Urszuli Szuścik, Danuty Kocurek

Wydawnictwo Uniwersytetu Śląskiego ■ Katowice 2019

(6)

Redaktor serii: Publikacje Wydziału Etnologii i Nauk o Edukacji

Urszula Szuścik

Recenzent

Eugenia Rostańska

Redakcja: Mariola Massalska, Krystian Wojcieszuk Projekt okładki: Mariusz Bieniek

Korekta: Sabina Stencel Łamanie: Marek Zagniński

Copyright © 2019 by Wydawnictwo Uniwersytetu Śląskiego

Wszelkie prawa zastrzeżone

ISSN 0208-6336 ISBN 978-83-226-3730-2

(wersja drukowana) ISBN 978-83-226-3731-9

(wersja elektroniczna)

Wydawca

Wydawnictwo Uniwersytetu Śląskiego ul. Bankowa 12B, 40-007 Katowice

www.wydawnictwo.us.edu.pl e-mail: wydawus@us.edu.pl

Wydanie I. Ark. druk. 8,0. Ark. wyd. 7,0. Papier offset. kl. III, 90g Cena 29,90 (w tym VAT) Druk i oprawa: Volumina.pl Daniel Krzanowski

ul. Księcia Witolda 7–9, 71 -063 Szczecin

(7)

Wstęp (Urszula Szuścik, Danuta Kocurek) . . . . Erkki Nevanperä

Basic education and evaluation in Finland . . . . Bożena Marzec

Organizacja ewaluacji szkół jako formy nadzoru pedagogicznego w Polsce, Anglii i Portugalii. Studium porównawcze . . . . Tomasz Kopczyński

Ewaluacja jako mistyfikacja nadzoru procesu dydaktycznego . . . . . Iwona Łukasiewicz

Ewaluacja w oświacie – użyteczna i wspierająca? Z doświadczeń dy- rektora szkoły . . . . Iwona Tyrna-Łojek

Ewaluacja nauczyciela akademickiego – wybrane kwestie . . . . Barbara Bandoła

Ewaluacja wewnętrzna metodą profilu szkoły. W trosce o jakość pracy.

Gimnazjum nr 2 z Oddziałami Integracyjnymi im. ks. prof. Józefa Tischnera w Woli . . . . Małgorzata Maksim, Ewa Bobel

Raport wyników ewaluacji w zakresie upowszechniania czytelnictwa wśród uczniów na przykładzie Gimnazjum im. Jana III Sobieskiego w Żółkiewce Noty o autorach . . . .

Spis treści

7 11

33 49

61 77

89 105 125

(8)
(9)

Wstęp

Postęp w dziedzinie wiedzy pedagogicznej związany jest z procesem wychowania, nauczania i kształcenia młodego pokolenia. Widoczny dynamizm przeobrażeń oświatowych zachodzący w dobie globalizacji w XXI wieku i pojawiające się oryginalne koncepcje kształtują nową rze- czywistość edukacyjną i rodzą pożyteczne innowacje w zakresie praktyki oświatowej.

We współczesnym dyskursie życia publicznego często używa się okreś- lenia ewaluacja. Termin ten jest różnie definiowany. Zdaniem Jacka Piekarskiego trudno dziś mówić o jednoznacznej językowej wykładni ewaluacji, kłopoty zaś interpretacyjne i dynamika dyskusji toczących się wokół niej jedynie ten problem zaostrzają1. Jedna z pierwszych jej defi- nicji na gruncie polskim znajduje się w Słowniku ilustrowanym języka polskiego Michała Arcta z 1929 roku. Ewaluacja jest tam rozumiana jako

‘ocenianie, oszacowanie, określenie wartości’2. Jak pisze Bolesław Nie- mierko: „Ewaluacja jest zbieraniem informacji o warunkach, przebiegu i wynikach działania edukacyjnego w celu dokonania oceny wartości tego działania i uzasadnienia dotyczących go decyzji. Narodziła się jako teo- ria, nazywana obecnie podejściem zoperacjonalizowanych zadań w łonie ortodoksyjnego behawioryzmu, lecz – w przeciwieństwie do nauczania programowanego – nie zgasła wraz z jego zmierzchem, lecz pomyślnie

1 Cyt. za: J. Piekarski: Jakość wiedzy pod kontrolą – uwagi na tematy klątwy ewa- luacji biuroktatycznej. W: Ewaluacja a jakość edukacji. Koncepcje – doświadczenia – kierunki praktycznych rozwiązań. Red. G. Michalski. Łódź 2011, s. 29–30.

2 Zob. J.M. Michałek, M. Marciniak: Ewaluacja a rozwój nauczyciela i szkoły w kontekście pracy z danymi. W: Ewaluacja a jakość edukacji…, s. 54.

(10)

Wstęp 8

ewoluowała”3. Grzegorz Michalski zauważa, że w naukach o wychowaniu jako najważniejszy cel ewaluacji przyjmuje się dążenie do doskonalenia rzeczywistości pedagogicznej. Wykorzystuje się podczas tego procesu różne rodzaje ewaluacji, m.in.: formatywną, konkluzywną czy społeczno- polityczną4.

Michael Scriven stworzył pojęcie ewaluacji kształtującej (formatywnej, konstruktywnej) jako wartościowania programu szkolnego w toku jego powstawania. Jego przeciwieństwo, wartościowanie gotowego programu, zostało nazwane ewaluacją sumującą (sumatywną, konkluzywną)5.

Na szczeblu systemów szkolnych ewaluacja przybiera obecnie postać systemów odpowiedzialności edukacyjnej (educational accountability sy- stems). Składają się na nie regulacje prawne i organizacyjne, według któ- rych instytucje edukacyjne i ich kadry są rozliczane z osiągnięć uczniów6. Podjęta w publikacji problematyka ma na celu zwrócenie uwagi na za- dania związane z ewaluacją, a także na stopień efektywności działań na- uczycieli i instytucji szkolnych oraz narzędzi wykorzystywanych w tym zakresie. Tematem ewaluacji zajęło się ośmioro autorów, którzy w prezen- towanych szkicach ukazali teoretyczne i praktyczne aspekty tego zagad- nienia.

Teksty przybliżają ewaluację procesu edukacyjno-wychowawczego w  Finlandii (Erkki Nevanperä), Anglii, Portugalii (Bożena Marzec) i  w  polskich placówkach oświatowych. Autorzy zwracają uwagę także na ewaluację pracy nauczyciela i trenera (Iwona Tyrna-Łojek), ewalua- cję użyteczną i wspierającą (Iwona Łukasiewicz) oraz ewaluację nadzoru pedagogicznego (Tomasz Kopczyński). Egzemplifikacją rozważań teore- tycznych jest ewaluacja wewnętrzna przeprowadzona w dwóch gimna- zjach, przedstawiona przez Barbarę Bandołę, Małgorzatę Maksim i Ewę Bobel.

Różnorodność poruszanych zagadnień pozwala przybliżyć horyzonty działań związanych z ewaluacją skoncentrowaną wokół wyzwań i realiów obecnych czasów. Przedstawione zagadnienia nie wyczerpują podjętej te-

3 B. Niemierko: Czy diagnostyka edukacyjna i ewaluacja w edukacji zachowają odrębność? „Teraźniejszość – Człowiek – Edukacja” 2013, nr 4 (64).

4 G. Michalski: Wstęp. W: Ewaluacja a jakość edukacji…, s. 5–6.

5 Za: B. Niemierko: Czy diagnostyka edukacyjna i ewaluacja w edukacji…

6 Ibidem.

(11)

Wstęp 9 matyki, ale mogą zainicjować głos w dyskusji dotyczącej różnych aspek- tów ewaluacji i dylematów związanych z omawianym zagadnieniem oraz być inspiracją do dalszych badań w tym zakresie. Należy mieć nadzieję, że praca będzie przydatna pedagogom, pracownikom instytucji eduka- cyjnoopiekuńczo-wychowawczych oraz studentom kierunków peda- gogicznych.

W tym miejscu pragniemy podziękować wszystkim autorom, którzy dzieląc się swoimi przemyśleniami, przyczynili się do powstania niniej- szej publikacji.

Urszula Szuścik, Danuta Kocurek

(12)
(13)

Erkki Nevanperä

Basic education and evaluation in Finland

Introduction

In Finland, the basic teaching has both educational and learning function. The aim is to support the students’ growth as individuals and as members of society, as well as to teach the necessary knowledge and skills. Basic education provides to all the same eligibility for further studies and it is free of charge. In addition to teacher’s books and other teaching materials, as also other tools and materials are provided free of charge.

Pupils are served, during every school day, a free full meal, and under certain conditions, free transport to and from a school. Pupils have also the right to get free pupil welfare services required for participation in education. Pupils with disabilities have the right to obtain free of charge all the aids and assistant services. Compulsory education for children and young people in primary education is usually given in elementary school. Among others, high schools for adults and folk high schools provide education for persons over the age of 18.

Primary school includes grades 1–9, and it is intended for the en- tire age group (7–16-year-olds). The class teacher, as a rule, gives class instruction during the initial six years of teaching. The class teacher teaches all or most of the school subjects. Three top-class education is primarily subject teaching. In Finland, there are about 3,200 schools that provide primary education. The majority of schools are communal.

The overall allotment of lesson hours for basic education and the minimum number of lessons for core subjects during basic education

(14)

Erkki Nevanperä 12

are decided by the government. The present allotment of lesson hours was enacted in 2012 and has been implemented together with the new core curriculum in August 2016. The distribution of lesson hours stipu- lates the core subjects taught to all pupils, and the distribution of teach- ing hours between various subjects.

The national core curriculum is determined by the Finnish National Board of Education. It includes the objectives and core content of dif- ferent subjects, as well as the principles of pupil assessment, special- needs education, pupil welfare and educational guidance. The principles of a good learning environment, working approaches as well as the con- cept of learning are also addressed in the core curriculum. The present national core curriculum for basic education was confirmed in January 2004 and it was introduced into schools in August 2006.

The education providers, usually the local education authorities and the schools themselves, draw up their own curricula for pre-primary and basic education within the framework of the national core curricu- lum. These curricula may be prepared for individual municipalities or institutions or include both sections.

The organizer of education is responsible for assessing its training and its effectiveness as well as to participate in activities’ external evalu- ation. The purpose of evaluation is to develop training and improve the conditions for learning. The monitoring of the local curriculum implementation and school year plan, and regular evaluation and de- velopment are part of this task.1

Since 1996, educational effectiveness has been understood in Fin- land to include not only subject-specific knowledge and skills but also the more general competences which are not the exclusive domain of any single subject but develop through good teaching along a student’s educational career. Many of these, including the object of the present assessment, learning to learn, have been named in the education poli- cy documents of the European Union as key competences which each member state should provide to their citizens as the part of general education.2

1 Basic Education Act 21 § subsection 1 and 2 (1296/2013).

2 EU 2006, Official Journal of the European Union. Recommendation of the Euro- pean Parliament and of the Council of 18 December 2006 on key competences for

(15)

Basic education and evaluation in Finland 13 The Finnish school system enjoys good reputation all over the world. Finnish 15-year-old pupils have been very successful for over 10 years, in the OECD’s PISA studies (OECD Programme for Inter- national Student Achievement).3 Finnish youth raised our country to the top of international education with great success already in the initial 2000-PISA study that assessed the three following areas: read- ing, mathematics, and science. The success has been since repeated in subsequent PISA surveys (2003, 2006, and 2009). The head of the study area renewed every three years. Therefore, the content of sufficiently broad and reliable comparison is possible only when a certain area of expertise is the subject of investigation for the second time. This was the case in the 2009 PISA study in 2000 estimated the first time in terms of the category of literacy. Results in mathematics were shifts in 2012.4

According to the OECD PISA 2012 study, know-how and standings of Finnish 15-year-olds in mathematics, reading, and science have fallen.

The ranking position of Finnish school education is fifth internationally, and number one in Europe. Differences between regions and schools are small. Skills of school boys within the scope of mother tongue are gen- erally weak. Gender differences become even more pronounced in other respects: only 30% of boys read monthly anything else than textbooks, whereas school girls – 60%. About 60% of girls and 40% of the boys wants to become better at language. Students appreciate mathematics.

The level of competence has fallen slightly in all areas of mathematics, such as geometry and percentage decline. The girls are inferior to boys in mental arithmetic. The students do not like maths, but both boys and girls think that maths is a useful skill.

lifelong learning (2006/962/EC), available at: http://eur-lex.europa.eu/LexUriServ/Lex UriServ.do?uri=OJ:L:2006:394:0010:0018:en:PDF.

3 According to J. Välijärvi, S. Sulkunen, PISA09, young Finns were again among the best in the OECD countries. The Ministry of Education and Culture & University of Jyväskylä 2010, Institute for Educational Research.

4 J. Hautamäki, S. Kupiainen, J. Marjanen, M-P. Vainikainen, R. Hotula- inen: “Oppimaan oppiminen peruskoulun päättövaiheessa, tilanne vuonna 2012 ja muutos vuodesta 2001.” Käyttäytymistieteellinen tiedekunta; Opettajankoulutuslaitos Tutkimuksia, no. 347. University of Helsinki 2013.

(16)

Erkki Nevanperä 14

Table 1. Credits and standings of Finnish 15-year-olds (PISA 2012) The results of Finland Credits OECD countries All participants

Mathematical skills 519 6 12

Literacy 524 3 6

Scientific literacy 545 2 5

Problem solving 523 4 9

Source: Author’s own elaboration.

Finland took part in the spring of 2011 for the first time since the 1999 in the TIMSS study of IEA (Trends in International Mathemat- ics and Science Study). The TIMSS study was focused on pupils of the 8th grade. The 1999 survey was conducted for seven classes, therefore, an additional sample was taken in Finland for comparison in this class degree. The TIMSS is different from the PISA mathematics assessment, because it is based on a test class education curriculum. In contrast to PISA, students’ skills are clearly fallen despite the fact that the 8-grad- ers’ knowledge level was the same as in other Western countries.

Kupari, Vettenranta and Nissinen (2012) describe a 38-point decrease in 7th-grade skills (520 > 482) as follows: “[…] it can be said that the 7th-grade maths skills had deteriorated during 12 years over one school year. Boys’ scores were reduced by 3 points more than the girls’ scores.”5

In spring 2012, the Helsinki University Centre for Educational Assess- ment implemented a nationally representative assessment of 9th-grade students’ learning to learn competence. The assessment was inspired by signs of declining results in the past few years’ assessments (the discipline- specific assessment of Board of Education and the OECD PISA 2009 study). The same schools which had participated in the respective na- tional assessment in 2001 were selected.

According to Hautamäki et al. (2013) the main focus of the assess- ment was on students’ competence and their learning-related attitudes at the end of the comprehensive school education, but the assessment also relates to educational equity: to regional, cross-school, and cross-

5 PISA 2012: The results of Finland, available at: http://www.minedu.fi/pisa/2012.html.

(17)

Basic education and evaluation in Finland 15 class differences and to the relation of students’ gender and home back- ground to their competence and attitudes.6

Girls performed better than boys in all three fields of competence measured in the assessment: reasoning, mathematical thinking, and reading comprehension. The difference was especially noticeable in read- ing comprehension even if in this task girls’ attainment had declined more than boys’ attainment. The impact of students’ home background was, instead, obvious: the higher the education of the parents, the better the student performed in the assessment tasks. There was no difference in the impact of mother’s education on boys’ and girls’ attainment.7

The change between the year 2001 and the year 2012 was significant.

The level of students’ attainment haddeclined considerably. The differ- ence can be compared to a decline of Finnish students’ attainment in PISA reading literacy from the 539 points of PISA 2009 to 490 points, to below the OECD average. The girls’ attainment has declined more than the boys’ in three out of five tasks. There was no gender difference as to the change of students’ attitudes, however. The cross-school differ- ences remained the same but differences between classes and between individual students increased.8

The change in attitudes – unlike the change in attainments – was related to students’ home background, namely, the decline in supportive attitudes and the growth in attitudes detrimental to school work were smaller if student’s mother was better educated. A decline could be discerned both among the best and the weakest students. It seems that the school’s role in young people’s lives is diminished.9

UNICEF has studied the equality of children in different countries (Fairness for Children 2016). The report shows that Finland fares well in the overall comparison. Finland came second in the said study, along with Norway and Switzerland. The ranking of our neighbouring coun- try Sweden had dropped tremendously. On the other hand, the results

6 P. Kupari, J. Vettenranta, K. Nissinen: “Kahdeksannen luokan oppilaiden matematiikan ja luonnontieteiden osaaminen.” Kansainvälinen TIMSS-tutkimus Suo- messa. Jyväskylä 2012: Koulutuksen tutkimuslaitos.

7 Ibidem.

8 Ibidem.

9 Ibidem.

(18)

Erkki Nevanperä 16

from Finland were suddenly the weakest in equality of education. The differences among Finnish 15-year-olds between average and weakest pupils were great at maths, reading, and conceptual thinking. The dif- ference increased from 2006 to 2012, approximately by 22%. For example, in reading this difference corresponds to the competence of third school year.10

The renewal of the core curriculum for pre-primary and basic education 2016

The national core curriculum for pre-primary and basic education has been renewed. The process involves all stakeholders, particularly education providers and education personnel. The aim is to also en- courage parents and pupils to participate in the process. The new lo- cal curricula that are based on this core curriculum should have been prepared by the beginning of school year 2016–2017.

The national core curriculum and local curriculum in Finland

The National Board of Education has approved the national core curriculum of basic education on 22nd December 2014. The local cur- riculum in accordance with these criteria will be introduced in grades 1–6 as of 1st August 2016. The purpose of basic education system is to ensure the equality and quality of education, as well as to create good conditions for the growth, development, and learning of students.

The basic education law and regulations, government decrees, national core curriculum and local curriculum and the plans of school year form the foundation for the control system. The different parts of the system are renewed in order to organize the teaching unable to take account of changes in the world around schools, and to strengthen the school’s role in building a sustainable future.

10 Ibidem.

(19)

Basic education and evaluation in Finland 17 The core curriculum is drawn up based on the basic education act and decree. The government decree determines the objectives and allotment of lesson hours.11 The document establishes a nationwide order issued by the Board of Education, according to which the local curriculum will be drawn. The task of national core curriculum is to support and guide the organization of teaching and school work and to promote the equal realization of unified basic education.12

Basic education is the entirety of teaching and education, where the objectives and content of different areas join together and form the basis for teaching and operating culture. As a result, the core curriculum shall contain provisions on objectives and content. The grounds contain, as far as necessary, also references to the law.

The local curriculum is the important part of control system. It plays a  key role in the implementation of national objectives and local objec- tives and tasks. The local curriculum will create the common basis and direction for daily school work. It is a strategic and pedagogical tool that aligns the activity of an education organizer, as well as the work of schools.

The curriculum connects the activity of schools to other local activities, which promotes the well-being and learning of children and young people.

The principles guiding the preparation of the local curriculum

The education provider is responsible for the preparation and devel- opment of the local curriculum.13 In the curriculum shall be decided basic educational work, teaching, assessment and learning support, guidance and pupil welfare, school and home cooperation, as well as other activities of the organization and implementation.

11 UNICEF Office of Research: “Fairness for Children: A league table of inequality in child well-being in rich countries.” Innocenti Report, Card 13, UNICEF Office of Research 2016 – Innocenti, Florence, available at: https://www.unicef-irc.org/publica tions/pdf/RC13_eng.pdf.

12 Basic Education Act 14 § subsection 2 and the Government decree (422/2012) 13 §.

13 Basic Education Act 15 § subsection 1.

(20)

Erkki Nevanperä 18

The evaluation and development of the local curriculum

The purpose of evaluation is the development of training and improv- ing the conditions for learning.14 The monitoring of implementation of the local curriculum and plan for the school year, regular evaluation and development are the parts of this task.

The organizer of education and school self-evaluation may take ad- vantage of the results of national assessments and development projects, as well as the national quality criteria15 of basic education. Cooperation with students, parents and other actors promote an open and constructive self-evaluation.

Changes in the core curriculum will also require making changes to the local curriculum. The organizer of teaching can check their cur- ricula and improve the quality and functionality also from local needs and utilize the results of the development work.

The assessment of learning in Southern Ostrobothnia and Teuva

The evaluation is based on the criteria set out by the curriculum objectives. The evaluation concentrates on student’s learning, work, and behaviour. Reliable evaluation requires the versatile observation and documentation of these areas.

The pupils and their performance cannot be compared with one an- other and the evaluation of students’ personality, temperament, or other personal characteristics. Teachers ensure that parents know the object- ives and evaluation criteria for pupils.

The evaluation is carried out on the consistent basis in a given province, so that the students are equal. The entrepreneurial approach

14 Quality Criteria of Basic Education, Ministry of Education and Culture 2012, p. 29.

15 16 Curriculum of the basic education of Teuva municipality 2016; available at:

http://www.teuvanyhtenaiskoulu.fi/teuvaops2016/teuvaops2.html.

(21)

Basic education and evaluation in Finland 19 and inclusion are regional priorities, which are taken into account in the evaluation.

The principles of evaluation

The evaluation is in line with the student’s age. The evaluation is a  regular, diversified, and goal-oriented. The task of evaluation is to guide and support the learning process and to promote the learn- ing to learn. The several tools of evaluation are presented in Table 2.

The duration and modalities of methods may vary.

Table 2. The principles of evaluation and some examples for implementation The principle of evaluation and implementation examples In addition in Teuva

1 2

Implication

•  self-assessment

•  students set their own objectives and assess their achieve-

•  portfolioment

•  peer review, pair and group assessment

•  interim evaluations;

discussion during the learning process

Aim to use regularly; the natural part of learning process.

Continuous observation and feedback as a part of daily instruction, e.g.

in connection with the amendment of tasks Incentive

•  attention to the whole learning process

•  realize even small advances

The entrepreneurial mode of operation

•  pupils set their own goals

•  aimis must be realistic / sub-targets

•  pupils  use  appropriate  assessment  practices  for  them- selves

•  pupils reflect on their own learning and school informs  aims and evaluation methods clearly to pupils and parents

•  school motivates students to try their best by the way of  encouraging assessment

Attention is paid to possi- ble barriers to learning.

For use in small, suppor- tive comments for essays, test papers, etc.

(22)

Erkki Nevanperä 20

1 2

Versatility

•  evaluation discussions

•  peer review, pair and group assessment

•  multidisciplinary learning modules

•  self-assessment in writing

•  certificates 

•  portfolio

•  drama 

•  visual expression

•  the evaluation of project

•  throughout the evaluation of the learning process

•  information technology is used to help

In addition to certificates, assessment discussions and multi-disciplinary learning modules, a tea- cher chooses appropriate evaluation methods for class. A teacher uses test as a learning event

The promotion of learning

•  continuous assessment during learning process

•  the encouragement and support of group or teacher

•  limited and clear aims Justice and ethicality

•  estimator  does  not  compare  students,  but  assesses  a student’s progress

•  the visualization of aims and evaluation (e.g. school in- forms aims and evaluation clearly to parents)

Practicing before a test.

After a test tasks are pas- sed through with discus- sion.

The purpose of evalua- tion for learning.

The development of emo- tional skills

Truthfulness

•  School  also  shows  the  need  of  development  issues  in supportive spirit

•  direct feedback

•  the increasing resilience of failure and disappointment The cooperation with parents

•  aims and evaluation are informed in advance to parents

•  other meetings

•  bulletins

•  parents’ evenings (informatively, e.g. targets go through)

•  evaluation discussions

•  clear  information  when  changes  are  taking  place  in learning

•  feedback from parents to school about the achievement  of aims, e.g. in evaluation discussions

•  school’s homepage, for example, behavioural criteria

Do not assess person, but a thing for learning.

Students are supported to identify their own strengths and weaknes- ses.Reasons for an evalua- tion, school practices, as well as curricular obje- ctives are shown in an  autumn announce- ment, as well as in parents’

evenings. Objectives can be found on schools’ web- site.

Source: Author’s own elaboration.

Table 1 continued

(23)

Basic education and evaluation in Finland 21

Pupil’s self-assessment and peer review

The self-assessment increases the self-awareness and self-confidence of children and adolescents. The self-assessment should be the everyday part of school work, realistic, reasonable and timely. Student’s under- standing of themselves as learners will be strengthened. The peer review directs students to evaluate joint or each other’s work and complete outputs orally or in writing. Via the peer review can also be assessed, how the group has achieved the aims set.

Pupils are taught already in elementary school to perform self-evalu- ation and peer review as the part of their learning and their education.

It is continued at secondary school.

Figure 1. Self and peer evaluation

Aims of Learning and Progress

Teacher tells clearly aims of subject / project/ working in accordance with a student's level - Student will set his/her own aims

- Teacher/ school informs aims to guardian Student’s self-assessment and peer review - It guides students to act self-directedly

- Student will identify his/her strengths, successes and areas which need development - It shall promote pupils' mutual evaluation and peer review discussion

- Self-assessment and peer review can be carried out:

- by discussing

- in pairs / in groups / alone - In writing

- as part of the portfolio - through drama

- by means of visual expression Learning

- Knowledge and skill of subject - goal-orientation - learning to learn (Identification and development of their own

learning styles)

Working - Team working skills - Communication skills - Taking care of their own

affairs - Regularity and versatility of

learning f - Goal-orientation

Behaviour - Consideration of another person

- Good manners -Mutually agreed procedures and

rules - Appropriate behavior for

situation

Student's understanding of himself/herself as a learner strengthens Source: Author's own elaboration

Self-evaluation forms are collected in a common folder on paper and electronic versions in Teuva. Teachers can choose suitable way for their

(24)

Erkki Nevanperä 22

own use. Documents are stored in paper or electronic folders. The reten- tion period of tests is at the end of the following semester.

Certificates

In addition to a school year, school will provide an evaluation feed- back at least once during a school year. Feedback may be an interme- diate certificate, an evaluation announcement or an evaluation debate.

A school year and an intermediate certificate shall comply with the guidelines of the national core curriculum and the Basic Education Regulation 852/1998. A school must be drawn up from the rest of evalu- ation feedback signed written document.

Certificates and evaluation forms are collected in student’s evaluation folders. These are kept in archives in accordance with the rules of filing.

The school-year evaluation

A verbal assessment is used for school year certificate in the grades 1‒3. The number evaluation of school year may be supplemented by the verbal evaluation in the grades 4–6. Art and practical subjects in the grades 4‒6 can be assessed only verbally in school year certificate as decided by the education provider. Student, parents and teachers must be able to interpret a certificate so that a learning progress and a competence should be clearly displayed.

The intermediate evaluation

The intermediate evaluation is carried out as the assessment dis- cussion in the grades 1–5. A student and parents are also present.

The objective of intermediate evaluation is to support and promote student’s learning and growth. A teacher carries out the evaluation.

A teacher will discuss with other class teachers on how a student has made progress. This feedback is transmitted via the evaluation debate to a student and parents. The basis for an evaluation debate is used

(25)

Basic education and evaluation in Finland 23 in the standard form in the grades 1 and 3–5. It is the same in the entire province and a  teacher records it. The assessment discussion is carried out in November through January, at the latest in fifth week. The assessment discussions are carried out in the cooperation time of home and school outside of class time. All who participate in discussion sign the document. The document will be archived in accordance with the education provider’s instructions. The original signed evaluation form is given to the guardians. If the parent does not arrive to the evalu- ation discussion, the teacher implements the evaluation together with the pupil and sends the evaluation to the guardian for information.

Second grade assessment discussion is carried out in the spring se- mester by week 12, so that it works at the same time as the evaluation of the joint phase (2).

The interim evaluations are carried out in Teuva as follows: in the grades 1 and 3–5 the interim evaluations are carried out in Novem- ber–January, and in the grade 2 during January and February. In the grades 1–5, before the evaluation discussion, guardians and a student pre-fill an evaluation form, on the basis of which the assessment discus- sion is taking place. The mid-term review is given on the basis of the evaluation discussion. In the year-class 6 pupils have a numerical mid- term review, which may be supplemented by a verbal assessment. The mid-term evaluation is carried out for the years 6–9 according to the same principle as the evaluation of school year. The quarts of parents and assessment discussions can be combined.

The evaluation of multi-sector learning modules

The multi-dimensional learning modules must be at least one per school year. This one-credit study module is also evaluated. The focus of the evaluation is on working skills.

The evaluation in a combined class

A teaching of a class can be done in full accordance with the pupils in grades or in part, in accordance with the principle of mutual

(26)

Erkki Nevanperä 24

exchange rates. The evaluation is carried out according to the objectives and content of the interactive course.

Progress of studies

A pupil progresses to the next class year, if a pupil has reached at least passable knowledge and skills of subjects. If a pupil lags behind in one of the subjects, teachers evaluate student’s potential success in the following year class studies. If a student is in danger of not passing class, the teacher must discuss it in suitable time with the student and the guardians. School will give the student an opportunity to demon- strate approved knowledge and skills in a separate experiment, or by any other display. School can also give the student an opportunity to do one subject-specific attempt in a separate experiment.

The learning unbound year categories

Pupils have possibility to move, if necessary, according to the progress of their own learning programme. Pupils can proceed according to their own study programme faster than by the way of year class learning or pupils can avoid the vacation of class.

The forms of evaluation by year classes

Pupil’s subjects of curriculum are evaluated numerically and verbally.

The verbal evaluation may also be the indication of performance. Elec- tive subjects (El.s) in the grades 4–6 are estimated verbally: pass/fail.

The elective lessons of art and craft subject (A & C- el.) are the part of common subjects as taught in music, visual arts, crafts, physical edu- cation and home economics education. The elective lessons of art and craft subjects are not evaluated in the school year certificate or in the leaving certificate. These are evaluated as the part of courses of common art and craft subjects for all.

(27)

Basic education and evaluation in Finland 25 Table 3. The forms of assessment by year-classes

Year-class / Forms of assessment Year-class / Forms of assessment

1 2

1st class

• self-assessment

• mid-term review: an assessment discus- sion pupil-guardians-teacher is carried out in November–January at the latest week 5

• standard form in the province (Annex 1)

• all involved in discussion are heard

• teacher acts as registrar

• evaluation of school year: a verbal evaluated at least mother tongue and lite- rature, mathematics, behaviour (separate attachment) and working skills

• multidisciplinary learning object is ta- ken into account assessment

• assessment discussion is carried out  3rd class

• self-assessment 

• mid-term review: an assessment discus- sion pupil–guardians–teacher is carried out in November–January at the latest week 5

• standard form in the province (Annex1)

• all involved in discussion are heard

• teacher acts as registrar

• an evaluation of school year: a verbal

•  evaluated  at  least  mother  tongue  and  literature, mathematics, A1 language, behavior (separate attachment) and working skills

• Multidisciplinary learning course is ta- ken into account in an assessment

2nd class

• self-assessment

• mid-term review/an evaluation in joint  point: an assessment discussion pupil- guardians-teacher is carried out in No- vember–January at the latest week 12

• standard form in the province (Annex 2)

• all involved in discussion are heard

• teacher acts as registrar

• evaluation of school year: a verbal evaluated at least mother tongue and lit- erature, mathematics, behavior (separate attachment) and working skills

• multidisciplinary learning object is ta- ken into account assessment

4th class

• self-assessment 

• mid-term review: an assessment discus- sion pupil-guardians-teacher is carried out in November–January at the latest week 5

• standard form in the province (Annex1)

• all involved in discussion are heard

• teacher acts as registrar

• an evaluation of school year: numeric,  which can be supplemented with words

• evaluated at least mother tongue and li- terature, mathematics, A1 language and behavior

• working skills are assessed in the con- text of school subjects, arts and crafts subjects can be evaluated numerically and/or verbally, elective substances are estimated pass/fail

• multidisciplinary learning course is ta- ken into account in an assessment

(28)

Erkki Nevanperä 26

Table 3 continued

1 2

5th class

• self-assessment 

• mid-term review: an assessment discus- sion pupil–guardians–teacher is carried out in November–January at the latest week 5

• standard form in the province (Annex1)

• all involved in discussion are heard

• teacher acts as registrar

• an evaluation of school year: numeric,  which can be supplemented with words

• evaluated at least mother tongue and li- terature, mathematics, A1 language A2 language and behavior

• working skills are assessed in the con- text of school subjects, arts and crafts subjects can be evaluated numerically and/or verbally, elective substances are estimated pass/fail

• multidisciplinary learning course is ta- ken into account in an assessment

6th class

• self-assessment

• mid-term review: numeric which can be  supplemented with words

•  all  subjects  are  evaluated  from  which  arts and crafts substances can be evalu- ated verbally

• behavior is  assessed too 

• elective substances are estimated pass/

• an evaluation in a joint point: a teacher fail gives guardians student feedback and informs about progress of learning in due time before the end of the school

•  an  evaluation  information  of  joint year point will be integrated as part of tea- ching organizer’s evaluation form

•  the  evaluation  of  the  school  year:  nu- meric, which can be supplemented with words according to the evaluation crite- ria of the curriculum (Grade 8)

• multidisciplinary learning course is ta- ken into account in an assessment

Source: Author’s own elaboration.

Table 4. The types of evaluation in some ways of teaching organizing Organizing way of teaching /

evaluation forms Organizing way of teaching / evaluation forms

1 2

Enhanced support

•  learning  evaluation  will  be  based  on  the general objectives of the core cur- riculum

Individualized school curriculums

• self-evaluation  is  always  assessed  in  accordance with the curriculum of PIOE (Plan of Individually Organized Education), verbally or numerically

(29)

Basic education and evaluation in Finland 27

1 2

Learning via operating area

• self-assessment, verbal review Teaching for year-classes unbound

• self-evaluation

• learning  modules,  which  are  entered  and completed in the study program are estimated

• school year certificate is issued for per- formed learning modules

• the  school  leaving  certificate  will  be  issued when the whole basic education syllabus is completed

Flexible basic education

• education provider decides in accordan- ce with the evaluation of a flexible basic  education curriculum

Teaching in hospital

• self-assessment

• home  school  provides  the  evaluation  of short period learning taking into ac- count the feedback of evaluation from hospital school

• at least the school year-long learning is  evaluated by hospital school

• school leaving certificate is given as the  form of home school and the principal of home school signs the certificate

Teaching in hospital (continued)

• home  school  and  hospital  school teachers together perform the final as- sessment

Teaching for immigrants

• self-assessment

•  assessing  the  learning  contents  of  the  curriculum verbally or numerically

• school  evaluates  the  management  of  substance, not Finnish language skills the competence of core corresponds the grade 8

• the final assessment numerical 

•  assessment  methods:  written,  oral  ex- pression, facial expressions, drawings

Source: Author’s own elaboration.

The evaluation of joint points in the grades 2 and 6

An evaluation discussion for the second class supports the evaluation of a joint point. In the evaluation discussion should take into account particularly the progress in language abilities, working skills and car- ing of pupils’ own tasks. Cooperation with parents is emphasized in the joint point. The evaluation form of intermediate and joint point is filled in the evaluation debate. All involved in discussion are heard,

Table 4 continued

(30)

Erkki Nevanperä 28

a teacher acts as registrar. The emphasis is in student’s self-evaluation.

The amount recorded in the affairs is not limited. All participants of discussion will sign the document and the document is archived ac- cording to the instructions of the education provider. A copy of the completed evaluation form is also given to parents.

A teacher gives guiding feedback for student at the end of sixth grade.

A pupil’s progress in working and learning skills are subsequently re- ported to parents. The joint point is a part of pupil’s transfer process from primary school to secondary school. It is evaluated the development of working, thinking and learning skills, the progress of subjects and wide- ranging know-how, as well as the level of motivation. The evaluation data of the joint point will be integrated into the evaluation form for transfer process. Information shall be communicated to parents. Transfer of data for the upper level is used already existing forms of municipalities.

The evaluation and the evaluation criteria of behaviour

Evaluation discussions held in schools are the basis for behavioural evaluation, taking into account school’s educational objectives, working culture of community and rules of conduct. Pupils and parents will be given an opportunity to attend school’s value discussion. The evalua- tion of behaviour should not be based on personality, temperament or personal characteristics of the student. The evaluation of behaviour is not the part of the evaluation of the subject. The main targets for the evaluation of behaviour are that a pupil notices other persons, good manners, jointly agreed mode of actions and rules, as well as appropri- ate behavior in the situation.

The evaluation of the behaviour follows the same format as the evalu- ation of subjects. The evaluation of pupil’s behaviour can be attached to the certificate. The annex will not be mentioned in the certificate.

The identity of the pupil must be included to the annex of mid-term and final evaluation.

The evaluation of behaviour is given in a separate appendix of school year certificate in Teuva. The evaluation in the grades 1–3 is identical in content. The evaluation of the pupil’s behaviour is based on self-assess-

(31)

Basic education and evaluation in Finland 29 ment in the grades 4–8. The evaluation is given in school year certificate in the grades 4–5. The evaluation of behavior is given both in the mid- term evaluation and in the school year certificate in the grades 6–8.

Table 5. Behaviour

Behaviour Separate appendix Appears on certificate

1 X

2 X

3 X

4 X

5 X

6 X

7 X

8 X

9 It may be annexed to the certificate Table 6. Criteria for grading behaviour

Grade Criteria

1 2

10Excellent • Pupil complies with school rules.

• Pupil  behaves  in  exemplary  politely,  friendly  and  helpfully  towards  others.

• Pupil behaves in exemplary fairly and tolerantly towards others.

• Pupil  adapts  to  work  in  different  groups  and  promotes  common work.

• Pupil promotes peace.

• Pupil’s language is exemplary.

• Pupil will create a positive atmosphere around by way of behavior.

• Pupil has not received a written warning and pupil has not received  disciplinary penalties.

9Creditable • Pupil complies with school rules.

• Pupil  behaves  in  exemplary  politely,  friendly  and  helpfully  towards  others.

• Pupil behaves in exemplary fairly and tolerantly towards others.

• Pupil adapts to work in different groups.

• Pupil gives peace to others.

• The student’s manner of speaking is effective.

• Pupil  is  not  stretched  out  and  pupil  has  not  received  disciplinary penalties.

(32)

Erkki Nevanperä 30

1 2

8Good • Pupil complies with school rules.

• Pupil adapts normally to work in different groups.

• Pupil gives normally peace to others.

• Pupils behave friendly towards others. 

• Pupil’s manner of speaking is usually effective.

• Pupil has been reprimanded, so that he will change his or her behavior  in guided direction.

7Satisfactory • Pupil has difficulties in complying with school rules.

• Pupil sometimes disturbs working of class.

• Pupil has some problems adjusting to work in different groups.

• Pupil has sometimes difficult to take others into account.

• Pupil’s language use requires guidance.

• It is necessary to instruct pupil’s behavior

• Pupil has received some disciplinary penalties.

6Moderate • Pupil has difficulties in complying with school rules.

• Pupil has problems adjusting to work in different groups.

• Pupil has difficult to take others into account.

• Pupil repeatedly disturbs working of class.

• Pupil’s language use often requires guidance.

• It is necessary to instruct pupil’s behavior repeatedly.

• Pupil has received a number of disciplinary punishments.

5Passable • Pupil has often difficulties in complying with school rules.

• Pupil has problems adjusting to work in different groups.

• Pupil does not give work in peace to others.

• Pupil does not able to take others into account.

• Pupil’s language use is inappropriate.

• It is necessary to constantly instruct pupil’s behavior.

• Pupil has received a number of disciplinary punishments, which do  not have an impact.

• Pupil’s behaviour is indifferent.

4Abandoned • Pupil is unable to act in the school community.

Source: Author’s own elaboration.

Table 6 continued

(33)

Basic education and evaluation in Finland 31

Final evaluation

The role of the final evaluation is to determine how the pupil has reached the objectives of syllabus at the end of study. The final assess- ment will take place according to the provincial distribution of lesson hours for the grades 7, 8 or 9.

Pupil’s final evaluation is not based directly on the pupil’s previous course certificates, period certificates or school year certificates. Because the development of know-how is always cumulative, the final grade will be based on the formation to show the level of knowledge in relation to the syllabus objectives and final evaluation criteria at the end of pupil’s studies. The final evaluation criteria for the value of 8 are determined in the basic education curriculum. The final evaluation is mainly numeric, except in Table 4, in some ways of organizing teaching. These elective subjects, which form a coherent, a minimum of two hours per week syllabus during the year, are estimated in numbers. The elective subjects, which contain less than two hours per week during year, are estimated in words. Advanced studies may raise the rating of the subject. More detailed instructions on school-leaving certificate and form of leaving certificate are in the basic education curriculum. The education provider decides the layout of leaving certificate.

Szkolnictwo podstawowe i proces ewaluacji w Finlandii

Streszczenie

Piętnastoletni uczniowie fińscy osiąga bardzo wysokie wyniki w badaniach PISA prze- prowadzonych przez OECD. W ostatnich latach poziom osiągnięć uczniów zmniejszył się.

Zmiana, która zaszła pomiędzy rokiem 2001 a 2012, jest znacząca.

W Finlandii szkolnictwo podstawowe ma zadania zarówno wychowawcze, jak i edu- kacyjne. Krajowa podstawa programowa dla szkolnictwa podstawowego, określona przez fińską Krajową Radę Edukacji, została wprowadzona do szkół w sierpniu 2006 roku. Pod- stawa programowa oraz ewaluacja zostały znowelizowane w 2016 roku. Ewaluację prze- prowadza się w regionach na stałych zasadach. Regionalnymi priorytetami są podejście przedsiębiorcze oraz inkluzja, które są uwzględniane w ewaluacji.

(34)

Erkki Nevanperä 32

Zasady ewaluacji to między innymi przedsiębiorczy tryb działania, ciągłość, promocja kształcenia się, prawdomówność i współpraca z rodzicami. Szkoła musi dostarczać infor- macji zwrotnych dotyczących ewaluacji przynajmniej raz w roku szkolnym. Informacje te mogą być przekazywane poprzez certyfikat śródsemestralny, powiadomienie ewaluacyjne lub debatę ewaluacyjną. Stosuje się też samoocenę.

Słowna ocena opisowa stosowana jest na świadectwach w klasach I–III. Ocena cyfro- wa na koniec roku szkolnego może być uzupełniona ewaluacją opisową w klasach IV–

VI. Ewaluacja śródsemestralna ma miejsce w postaci debaty ewaluacyjnej w klasach I–V w obecności ucznia i rodziców. Dodatkowymi wyzwaniami w zakresie ewaluacji są między innymi elastyczna edukacja podstawowa, nauczanie szpitalne i edukacja imigrantów.

Słowa kluczowe: Finlandia, krajowa podstawa programowa, ewaluacja, podejście przed- siębiorcze, współpraca z rodzicami, inkluzja, debata ewaluacyjna, samoocena

Basic education and evaluation in Finland

Summar y

Finnish 15 years old pupils have been very successful in the OECD’s PISA tests. In recent years, the level of students’ attainment has declined. The change between the year 2001 and year 2012 is significant.

In Finland, basic teaching carries out both educational and didactic task. The national core curriculum for basic education is determined by the Finnish National Board of Edu- cation, and it was introduced in schools in August 2006. The amendments to the national core curriculum and the evaluation were introduced in 2016. The evaluation is carried out on a consistent basis in the provinces. The entrepreneurial approach and inclusion are regional priorities, which are taken into account in the evaluation.

The principles of the evaluation are, among others, incentives, entrepreneurial mode of operation, continuity, promotion of learning, truthfulness and cooperation with parents.

School must provide an evaluation feedback at least once during a school year. The feed- back may be given in the form of the intermediate certificate, the evaluation announce- ment or the evaluation debate. Self-assessment is also in use.

Verbal assessment is used for the school year certificate in the 1–3 grades. The grad- ing system used for the evaluation of school year, expressed in numerical form, may be supplemented by verbal evaluation in the 4–6 grades. The intermediate evaluation is car- ried out as an assessment discussion in the 1–5 grades, a student and the parents are also present. What is additional challenges for the evaluation are, among others, flexible basic  education, teaching in hospital and teaching for immigrants.

Key words: Finland, national core curriculum, evaluation, entrepreneurial approach, co- operation with parents, inclusion, evaluation debate, self-assessment

(35)

Bożena Marzec

Organizacja ewaluacji szkół

jako formy nadzoru pedagogicznego w Polsce, Anglii i Portugalii

Studium porównawcze

Wstęp

Nadzór pedagogiczny jako jeden z nielicznych nadzorów admini- stracyjnych znajduje bezpośrednie źródło w przepisach Konstytucji Rzeczypospolitej Polskiej1. Przepis art. 70 ust. 3 ustawy zasadniczej za- kłada nadzór pedagogicz ny nad szkołami i zakładami wychowawczy- mi oraz nakazuje uregulowanie zasad jego sprawowania w  ustawie zwykłej. Wytyczną zawartą w podanym przepisie realizuje ustawa z dnia 7 września 1991 r. o systemie oświaty2, która określa przedmiot nadzoru pedagogicznego, organy uprawnione do jego sprawowania oraz środki służące organowi sprawujące mu ten nadzór. Obowiązek określenia szczegółowych zasad sprawowania nadzoru pedagogicz- nego oraz kwalifikacji osób uprawnionych do jego sprawowania usta- wa nakłada w art. 35 ust. 6 na ministra właściwego do spraw oświaty i wychowania.

Nadzór pedagogiczny w Polsce

W październiku 2009 roku wydano czwarte od 1999 roku rozporzą- dzenie wprowadzające zasadnicze zmiany w systemie nadzoru peda-

1 Konstytucja Rzeczypospolitej Polskiej z dnia 2 kwietnia 1997 r. (Dz.U. 1997 Nr 78, poz. 483).

2 Ustawa z dnia 7 września 1991 r. o systemie oświaty (tekst jednolity: Dz.U. 2015, poz. 2156).

(36)

Bożena Marzec 34

gogicznego. Jedną z najważniejszych było wprowadzenie w struktu- ry nadzoru procesu ewaluacji jako podstawowej formy sprawowania nadzoru pedagogicznego3. W rozporządzeniu tym został ustalony wy- kaz stanowisk w  kuratoriach oświaty oraz urzędach innych organów sprawujących nadzór pedagogiczny i podporządkowanych im jed- nostkach organizacyjnych, których zajmowanie wymaga kwalifikacji pedagogicznych.

Zgodnie z rozporządzeniem nadzór pedagogiczny jest realizowa- ny przez wykonywanie zadań i czynności w trybie działań planowych lub doraźnych i ma służyć doskonaleniu systemu oświaty, w szcze- gólności podnoszeniu efektów pracy szkół. Dodatkowym celem jest ocena jakości działania szkół i placówek oświatowych oraz wspoma- ganie ich pracy w  dążeniu do uzyskiwania pożądanej jakości dzia- łalności dydaktycznej, wychowawczej i opiekuńczej oraz innej dzia- łalności statutowej. Podniesienie jakości systemu oświaty poprzez usprawnienie pracy szkół oraz wsparcie w rozwoju uczniów i nauczy- cieli stało się priorytetowym celem modernizacji nadzoru pedago- gicznego. W rozporządzeniu określone zostały jego formy, do których należą:

– ewaluacja działalności edukacyjnej szkół i placówek (rozróżniamy ewaluację zewnętrzną i wewnętrzną; ewaluacja prowadzona przez organ sprawujący nadzór pedagogiczny jest ewaluacją zewnętrzną, a przez dyrektora szkoły lub placówki we współpracy z nauczycielami – ewaluacją wewnętrzną);

– kontrola przestrzegania przepisów prawa dotyczących działalności dydaktycznej, wychowawczej i opiekuńczej szkół, placówek i nauczy- cieli;

– wspomaganie pracy szkół i placówek oraz nauczycieli w zakresie ich działalności dydaktycznej, wychowawczej i opiekuńczej oraz innej działalności statutowej.

3 Rozporządzenie Ministra Edukacji Narodowej z dnia 7 października 2009 r.

w sprawie nadzoru pedagogicznego (Dz.U. 2009 Nr 168, poz. 1324).

Cytaty

Powiązane dokumenty

33 Spätestens ab September 1943 ist zu vermuten, dass unter oder durch Steinmeyer Patienten direkt in der Anstalt Pfafferode aktiv ermordet wurden: Ab 8.. September 1943

Zatem we wszystkich metatekstowych uwagach odautorskich, które pojawiają się w powieściach tego gatunku, nie tylko eksponowany jest jego związek z konwencją powieści

After Berdyaev, miłosz identified that Russian ambivalence of the Eastern realisation of evil in Dostoevsky: whoever lies, spreads terror, rapes, and kills, they sin, but

Postawa przyje˛cia łaski zbawienia zakłada proces nawrócenia poje˛ty jako przejs´cie od naturalnego wysiłku człowieka, d ˛az˙ ˛acego do ocalenia swej ziemskiej egzystencji

Drużyn Bartoszowych czy Stałych Drużyn Sokolich, dostrzegając jedynie dorobek Związków Strzeleckich i Polskich Drużyn Strzeleckich, patrząc przez pryzmat tych organizacji

Koncepcja banku emisyjnego na ziemiach polskich została urzeczywistniona dopiero w okresie Królestwa Polskiego (nie licząc in- stytucji emisyjnej z okresu powstania kościuszkowskiego

A specific focus of the PISA project cyclic tests, see Table 1, was to obtain information for at least one pile at each site under a large number of load cycles (>25000), to

Zwłaszcza, że znajdują się tutaj obiekty osadnicze kultury pomorskiej ze środkowego okresu lateńskiego, niemające zbyt wielu analogii.. Możemy mieć do czynienia z