77
Improving publication strategies in architectural design science
Frank van der Hoeven
This chapter explores how improving publication strategies in the architectural design discipline can contribute to establishing architecture as a mainstream science in its own right.
In the fields of architecture, landscape architecture, and urban design, a long-standing debate on the relationship between science and design persists, fed by doubts on the scientific performance of design disciplines. When performance indicators of mainstream sciences are used to measure the research performance in design, those indicators may not even appear to exist. The Rathenau Institute (part of the Royal Netherlands Academy of Arts and Sciences) concluded that “there is no stable publication pattern nor a core set of scientific journals to make a valid bibliometric benchmarking of architectural departments”.1
However, research is most certainly conducted in the overlapping fields of architecture, land-scape architecture, and urban design, though the outcomes of those investigations are seldom published in peer-reviewed academic journals. In the rare cases that researchers publish archi-tectural investigation in refereed periodicals, they tend to do so alone: references to the work of others are sparse or even lacking. As a result, the number of journal articles per person and the frequency in which others cite these publications remain low. In mainstream sciences, these are the two primary indicators used to assess the quality and significance of researchers.
In the past, the lack of peer-reviewed publications has led to disappointing evaluations of re-search groups and individuals. Institutions that carry out rere-search assessments, together with organisations that provide research funding, have become aware of a so-called evaluation gap.2
They understand that traditional performance indicators do a below-average job of describing the performance of research groups and individuals in design sciences.
The evaluation of research groups in design has recently improved in the Netherlands through the use of peer-group assessment and an improved protocol3, although the evaluation of
indivi-dual research grants and research projects is, unfortunately, not improving. When indiviindivi-duals and groups are unable to obtain research funding, institutions—cutting budgets in austere times—can no longer support them. Therefore, this matter needs to be addressed, as funding issues are a driving force for change at this point.4
What happens from here? What actions can the architectural design discipline take to prevent its research from being undervalued? In what ways can the discipline be included in the scientific mainstream, without giving up the strengths that make it unique?
El Croquis
In the summer of 1991, I worked as an intern in a small architectural design office in Amsterdam, staffed by Ben (the architect), Caroline (the theorist), two regular staff members, some tempo-rary workers, and the usual interns. Ben had his mind firmly set on being the focus of an issue of El Croquis—the Spanish architectural design glossy that publishes dedicated issues on major contemporary architects—which would indicate that Ben had established himself as an
interna-78 79 ses. Rather, it primarily analyses the metadata of peer-reviewed publications, such as author,
affiliation, abstract, references, and keywords.
Citation indexes play a key role in bibliometrical analyses. The leading index in this field is Thomson Reuters’ Web of Science, previously known as ISI.6 Elsevier’s Scopus is the emerging
competitor. Access to the Web of Science and Scopus is based on (institutional) subscription, and thus the broader community of designers and the general public are effectively locked out of the use of these instruments. Google has developed its own scientific index called Scholar, which is free and therefore publicly accessible, though most mainstream scientists consider it to be less rigorous. For these scientists, to be listed in Google Scholar is not as significant as being listed in the Web of Science or Scopus. Because indexes gather and organise scientific informa-tion and provide search capabilities, they have become an important source of informainforma-tion for researchers.
Bibliometrical analyses tend to use the largest indexes that are available. Here, the Web of Science (ISI) dominates. About 40 architectural design journals are included in the Web of Science, and TU Delft has listed these on its website.7 Note that this list does not include
Archi-tectural Digest (AD), although it is included in the Web of Science. This collection of architec-tural periodicals is by no means comprehensive, which may explain why the Rathenau Institute was unable to find a stable publication pattern. Scopus tries to establish itself by focussing on those areas where the Web of Science is traditionally weak: social sciences and humanities. It in-cludes more periodicals that are relevant to the field of architecture, but its coverage still cannot match those of specific indexes like Columbia University’s Avery Index to Architectural Periodi-cals, the Design and Applied Arts Index (DAAI), and RIBA’s Architectural Publication Index (API). The Avery Index is the most comprehensive index among these three, with over 260 source titles listed, including El Croquis.8 The Avery Index differs from the Web of Science and
Scopus in that it does not index references and affiliations, and therefore it cannot be used for bibliometrical analyses. It does index designers, however —something the other indexes do not. For instance, a search on Ben van Berkel produces 482 results, with 334 hits on UN Studio. Rem Koolhaas harvests 1332 results, while OMA gets just 42.
tional architect of stature. In 1995, with the publication of El Croquis 72, Ben van Berkel achieved his goal. 5 A few years later, Van
Berkel & Bos evolved into UN Studio, and the rest is history.
Although El Croquis dedicated issue 72 ex-clusively to the work of Van Berkel & Bos, it should be understood that Ben Van Berkel did not publish work in that journal, though Caroline Bos did. El Croquis, on a par with peer-reviewed journals in mainstream scien-ces, is actually like many other publications in architecture: it assigns its own writers and editors to produce content. In other sciences, this would be out of the ordinary, and a major practitioner would not aspire to have his or her work published by someone else. Rather, he or she would be the one that writes the article, most likely with a substantial number of co-authors. The ‘author’ in mainstream sciences is the person who conducted the research; therefore, no author would dare to publish the work (or interpret the research) of another.
The way that designers deal with accomplishments, and corresponding credits, differs radically from mainstream scientists. Architects receive more esteem when someone else publishes their work than when work is self-published. But because designers refrain from publishing their de-sign work, others cannot refer to it, though they can refer to the authors that have written about that designer, and most certainly, they will refer to the buildings, public spaces, and landscapes that are the product of their designs. Many of the references that are used in this context are images, not words: images from buildings, models, sketches, technical drawings, and computer renderings. When images trump words, tools used in mainstream sciences to unearth the links between researchers and to measure their performance are relatively powerless in design fields.
Bibliometrics
The set of tools used to measure research performance are commonly referred to as bibliome-trics. These keep score of the publication behaviour of researchers and research groups through citation and content analysis. The aims of bibliometrics are to draw conclusions on the relation-ships between researchers and to measure quantitative aspects of their scientific performance. Through the analysis of journals (or any other material with an international standard serial number, or ISSN), the tools establish which serial numbers have impact, which institutions count, and which researchers stand out from the crowd. Bibliometric analysis is not conducted by reading the actual research output or by user feedback: it doesn’t depend on qualitative
analy-Figure 1: El Croquis 72 : Ben van Berkel, 1990-1995
80 81 journals and books have never prioritized such criteria, and a larger-than-average number of
journals did not pass the test. None of the journals of NAi010 Publishers were accepted. Publishers of architectural journals and books must perform better. Researchers and their institutions should support initiatives to improve the quality of relevant journals. The websites of journals could be improved with open-source software such as Open Journal Systems (OJS). Journals should include English-language abstracts. Model policies developed by the Committee on Publication Ethics could be easily adopted. Journals could simply require that authors start referring to one another. All of these are relatively simple changes that would make a difference.
Submit journals to indexes
Authors, institutions, and publishers need to work together to ensure that publications are suc-cessfully submitted to indexes, directories, and registries such as the Web of Science, Scopus, the Avery Index, the Directory of Open Access Journals, and the Open Archive registry. Journals must make their content and metadata available in a structured way (i.e. using Dublin Core, PKP metadata) so that Google Scholar can harvest it. OJS does a good job in this respect.
Provide Open Access
For decades, publications in architecture, landscape architecture, and urban design reached a large and diverse audience: among the readership were practitioners and even parts of the general public. Authors preferred publishing in journals with a wide distribution, even in those that sold single issues in major bookshops. Now, the limited availability and expensive sub-scriptions that characterise many of the academic peer-reviewed journals effectively lock out large parts of the readership. Limiting readership solely to researchers is counterproductive in a context where the societal relevance and economical effectiveness of the research becomes a determining factor.
The current development towards open access is helpful at this point. The Directory of Open Access Journals already lists 8,000 journals.12 Most open-access journals will ask the author, or
the author’s institution, for a fee to cover the publication, but never the reader. Universities and designers have a lot to gain by embracing this formula. Architectural design journals should transfer to open access.
Provide high quality graphic design
Architects are accustomed to a high quality of print and graphic design, and most academic jour-nals do a poor job in that respect. Academic jourjour-nals need to improve in this area. As explained earlier, images play a key role in referring to design work. The representation of the image is es-sential for the overall quality of a design-oriented journal, both in print and online.
Make design count
Journals that publish original design work and use a peer review to evaluate the quality of sub-missions are still rare. The development of this formula could take off once designers are able to present the references to design work (as listed in the Avery Index, for instance) as solid evi-dence of design competence. This issue needs to be resolved with funding agencies, and institu-tions should lead the way.
Initial response
The initial response from the design disciplines to the lack of peer-reviewed journal articles has been two-fold. On one hand, academics plead that design should be regarded as research output, rather than peer-reviewed journal articles. Other academics tried to break new ground delving into design methodology, design theory, and architecture criticism.9 So far, not much
has changed for the better. In some respects, the situation has become even grimmer, now that much of the cultural funding is being slashed under the pressure of the enduring economic crisis. Previously, this type of funding was one of the areas in which the design disciplines performed significantly better than other sciences.
Horizon 2020
Due to that same economic crisis and also to pressing environmental and societal challenges, more attention is being paid to the way that research can contribute to the competitiveness of industries and societal issues, so-called ‘grand challenges’ like employment, research and development, climate change and energy, poverty and social exclusion, and education.10 The
next research programme of the European Union, Horizon 2020, focuses on three areas: excel-lent science, competitive industries, and better society. If contributions to ‘competitive indus-tries’ and ‘better society’ were to be included in the evaluations of research groups and indivi-duals, the measured research performance of the architectural design discipline would signifi-cantly improve. Architectural design addresses the application of research findings in industry and society rather well, although work still needs to be done in regards to scientific excellence.
No simple solution
Too often, both individual grant proposals by young researchers and also research proposals by designers-turned-professors collapse as a result of the lack of evidence of scientific excellence. Here, design sciences compete in the much broader group of sciences, not just with other archi-tectural designers. In these assessments, the number of peer-reviewed journal articles (or book publications) and their corresponding citations will remain a determining factor. A breakthrough is required at this point. The solution seems simple at first glance: researchers and professionals in the field of architecture, landscape architecture, and urban design should publish more work in peer-reviewed journals, and perhaps less in trade journals. What is holding them back?
Improve publishing practices
The limited availability of architectural periodicals in indexes is often cited as an issue for designers wishing to publish work. Journals do not appear automatically in indexes. In fact, the Web of Science and Scopus require that a journal be submitted in order to be indexed. The publisher, editor, or author can request such an evaluation in a process that sounds easier than it actually is. When a submission request is received, for instance, by Scopus, Elsevier will contact the paper’s publisher and editors. In too many cases, those parties simply do not respond, and the submission process is never completed. When the publisher does respond by completing the online form and submitting three recent issues, the journal can be evaluated. The reviewers will look at qualitative criteria, including the review process, the editorial policies, the ethics paragraph, and the availability of English abstracts. A good overview of such criteria is provided by Scopus11. The final hurdle of the process is the citation analysis: an index wants
82 83
1 B. van der Meulen et al., 2011, Evaluating Research in Context: Pilot Study at the Faculty of Architecture TU Delft,
Final Report: Confidential, The Hague: Rathenau Institute, Online available from: http://repository.tudelft.nl/view/ir/ uuid%3Af0a713f1-1564-4b79-be66-4f5299ebba2c/ [Accessed: 1st March 2011].
2 NOW, 2010, Evaluating Research in Context, Online available from: http://www.eric-project.nl/nwohome.nsf/pages/
NWOA_6TZJ28_Eng [Accessed: 1st January 2011].
3 F. van der Hoeven, 2011, Mind the evaluation gap: reviewing the assessment of architectural research in the
Nether-lands, In Architectural Research Quarterly ARQ, 15(2), 177-187.
4 D. Strand, 1998, Research in the Creative Arts. Canberra: Department of Employment, Education, Training and
Youth Affairs, XV–XVI, Online available from: http://catalogue.nla.gov.au/Record/364383 [Accessed: 1st March 2011].
5 C. Bos & G. Lynn, 1995, Ben van Berkel, 1990-1995, In El Croquis, 72.
6 Thomson Reuters, 2012, Web of Science, Online available from:
http://thomsonreuters.com/products_services/sci-ence/science_products/a-z/web_of_science/ [Accessed 27th July 2012].
7 TU Delft Faculty of Architecture, 2012, Architecture Journals in Web of Science. Online available from: http://
bk.tudelft.nl/nl/onderzoek/publicaties/wos-architecture-journals/ [Accessed 27th July 2012].
8 Columbia University, 2012, Avery Index, Online available from: http://library.columbia.edu/indiv/avery/avery_index.
html [Accessed 27th July 2012].
9 T. Avermaete, C. Grafe, K. Havik, J. Lagae, Patteeuw., et al., 2010, Editorial - Constructing Criticism, In OASE
Jour-nal for Architecture, 81, 1-10.
10 European Union, 2012, Europe 2020, Online available from:
http://ec.europa.eu/europe2020/europe-2020-in-a-nutshell/targets/index_en.htm [Accessed 27th July 2012].
11 Elsevier, 2012, Content Selection, Online available from: http://www.info.sciverse.com/scopus/scopus-in-detail/
content-selection [Accessed 27th July 2012].
12 DOAJ, 2012, Directory of Open Access Journals, Online available from: http://www.doaj.org [Accessed 27th July
2012]. Blur the line between journals and books
Books are still favoured by the design community, and much that has been said about journals in this chapter accounts for books as well. Architectural book publishers must learn that books can be indexed as well, as long as sound editorial policies are applied and the books are provided with an ISSN. In the end, the difference between books and journals may even fade away.
Publication strategies count
The architectural design discipline can prevent its research from being undervalued by improving (the impact of) its publication strategies. Key actions in this respect are
improving publishing practices, submitting journals to indexes, moving towards open access, providing high quality graphic design, making design count, and blurring the line between journals and books. The question of how design disciplines could become more of the scientific mainstream—without giving up the strengths that make them unique—is easy to answer. The design sciences should strengthen their scientific excellence through improved publications strategies. Then, contributions to the economy and society will speak for themselves.
