Lefschetz coincidence formula on non-orientable manifolds by

153 (1997)

Lefschetz coincidence formula on non-orientable manifolds

by

Daciberg Lima G o n ¸c a l v e s (S˜ao Paulo) and Jerzy J e z i e r s k i (Warszawa)

Abstract. We generalize the Lefschetz coincidence theorem to non-oriented mani- folds. We use (co-) homology groups with local coefficients. This generalization requires the assumption that one of the considered maps is orientation true.

1. Introduction. The Lefschetz Fixed Point Theorem may be reformu- lated to detect coincidences of a pair of maps f, g : M → N between closed manifolds of the same dimension ([V], Ch. 6, [Bd], Ch. 6). Such a modi- fication, however, needs the orientability assumption on M and N . In the same chapter, page 176 of [V], the author says “It should be pointed out the spaces we consider, closed, orientable manifolds, could be made more gen- eral. Similar techniques may be applied in the nonorientable case by using twisted coefficients.” The aim of this work is to present such an extension.

So we drop the assumption that the manifolds are orientable and we only assume that one of the considered maps, say g, is orientation true, i.e. a loop α preserves a local orientation of M iff gα preserves a local orientation of N . However, without the above assumption on g such an extension does not seem possible.

In Section 2 we prepare some information on (co-) homology with coef- ficients in a local system. We use them (Section 3) to prove the Poincar´e duality and then to get the promised Lefschetz theorem (Section 4). In Sec- tion 5 we find a relation between the coincidence index defined there and the semi-index from [DJ] and [Je]. At the end of this section we sketch how to define an index for a Nielsen class without restrictions on the maps f and g. In Section 6 we prove a result about coincidence producing maps as defined in [BS].

1991 Mathematics Subject Classification: Primary 55M20; Secondary 57N99.

The first author has been supported by the international cooperation program GMD/Germany-Cnpq/Brasil and Universidade de S˜ao Paulo during his visit at the Math- ematische Institut der Universit¨at Heidelberg, where part of this work was done.

[1]

2. Homology with local coefficients. In this section we recall some basic information about homology with coefficients in a local system [Wh], [Sp2]. Let R be a commutative ring with unity, X a topological space and let Γ be a local system of R-modules over X, i.e. over each point x ∈ X we have a module Γ (x) and for each homotopy class ω of paths from x ₀ to x ₁ we have an R-module homomorphism Γ (ω) : Γ (x ₀ ) → Γ (x ₁ ). Let

∆ ∗ (X; Γ ) denote the chain complex with coefficients in Γ (i.e. the graded R-module of sums P

γ _σ · σ where σ is a singular simplex in X with γ _σ ∈ Γ (σ(v 0 )), where v 0 denotes the leading vertex of the standard n-simplex) and let H _∗ (X; Γ ) be its homology module obtained by using the boundary operators defined on local coefficients. Similarly we consider the cohomology H ^∗ (X; Γ ^∗ ) with coefficients in the dual system Γ ^∗ = Hom(Γ, R) (see [Wh], [Sp2] for details). Let Γ, Γ ⁰ be local systems over X and Y respectively.

To get a chain homomorphism induced by a map f : X → Y we need a morphism of the given local systems, i.e. a commutative diagram

Γ Γ ⁰

X Y

φ //

²² ²² _f //

where the restriction of φ to any fibre is a homomorphism and φ(ω(1)) · Γ (ω) = Γ ⁰ (f ω) · φ(ω(0))

for any path ω in X, where φ(x) is the homomorphism φ restricted to the fibre Γ (x). We define maps f _∗ : ∆ _∗ (X; Γ ) → ∆ _∗ (Y ; Γ ⁰ ) by f _∗ (γ · σ) = φ(γ) · f σ and f ^∗ : ∆ ^∗ (Y ; Γ ^0∗ ) → ∆ ^∗ (X; Γ ^∗ ) by (f ^∗ (c)(σ))(γ) = (c(f σ))(φγ).

Recall that there is a natural pairing (Kronecker index) h·, ·i : H ^q (X, A; Γ ^∗ ) ⊗ H _q (X, A; Γ ) → R.

The symbol ⊗ denotes tensor product over R. We will also denote by R the constant system with the fibre R.

In this paper Γ , Γ ⁰ , . . . denote one-dimensional local systems, i.e. each Γ (x) is an R-module isomorphic to R. Then the chain complexes ∆ ∗ (X ×Y ; Γ × Γ ⁰ ) and ∆(X, Γ ) ⊗ ∆(Y, Γ ⁰ ) are naturally isomorphic. This yields the following products:

• the cross product

× : H _p (X, A; Γ ) ⊗ H _q (Y, B; Γ ⁰ ) → H _p+q ((X, A) × (Y, B); Γ × Γ ⁰ ),

× : H ^p (X, A; Γ ^∗ ) ⊗ H ^q (Y, B; Γ ^0∗ ) → H ^p+q ((X, A) × (Y, B); (Γ × Γ ⁰ ) ^∗ ),

(provided X × B, A × Y is excisive in X × Y ),

• the cup product

∪ : H ^p (X, A; Γ ^∗ ) ⊗ H ^q (X, B; Γ ^0∗ ) → H ^p+q (X, A ∪ B; (Γ ⊗ Γ ⁰ ) ^∗ ),

• the cap product

∩ : H ^p (X, A; G) ⊗ H n (X, A ∪ B; Γ ) → H n−p (X, B; Γ ⊗ G)

(in the cup and cap products we assume that {A, B} is excisive in A ∪ B).

For future reference we list some of their properties:

2.1. Let Γ, Γ ⁰ be local systems on X and let x ∈ H ^p (X, A; Γ ^∗ ), y ∈ H ^q (X, B; Γ ^0∗ ). Then

x ∪ y = (−1) ^pq (y ∪ x).

2.2. Let x ∈ H ^p (X, A; Γ ^∗ ), y ∈ H ^q (X; R) and a ∈ H _p+q (X, A; Γ ). Then hx, y ∩ ai = hx ∪ y, ai.

2.3. Let x ∈ H ^p (X, A; R), y ∈ H ^q (Y, A ⁰ ; R), a ∈ H r (X, A ∪ B; Γ ) and b ∈ H _s (Y, A ⁰ ∪ B ⁰ ; Γ ⁰ ). Then

(x × y) ∩ (a × b) = (−1) ^p(s−q) (x ∩ a) × (y ∩ b)

∈ H r+s−p−q ((X, B) × (Y, B ⁰ ); Γ × Γ ⁰ ).

In the next section we will also use the following lemma (compare the formula (5.20) on p. 150 of [V]).

Lemma 2.4. Let M be an n-manifold with a compact boundary and fix U ∈ H ⁿ (M × M, M × M − ∆; (R × Γ ) ^∗ ) and x ∈ H ^p (M, ∂M ; R). Let U ∈ H e ⁿ (M × M, ∂(M × M ) − ∂∆; (R × Γ ) ^∗ ) be the restriction of U and let 1 ∈ H ⁰ (M ; R) denote the unit. Then

U ∪ (x × 1) = e e U ∪ (1 × x) ∈ H ^n+p (M × M, ∂(M × M ); (R × Γ ) ^∗ ).

P r o o f. It is enough to show that U ∪ (x × 1) = U ∪ (1 × x) ∈ H ^n+p (M × M, M × M − ˙ ∆); (R × Γ ) ^∗ ) (here ˙ ∆ = ∆ − ∂∆). Let us fix:

(i) a collar C = ∂M × [0, 1) ⊂ M ;

(ii) a neighbourhood V of ∆ ⊂ M × M such that the projections p 1 , p 2 : V → M are homotopic rel. ∆ (p _i (x ₁ , x ₂ ) = x _i );

(iii) a neighbourhood V 0 of ∂∆ ⊂ M × M such that V 0 ⊂ V ∩ (C × C), p ₁ (V ₀ ) ∪ p ₂ (V ₀ ) ⊂ C and the above homotopy is a homotopy of pairs (V, V ₀ ) → (M, C);

(iv) a collar C ₁ of ∂∆ ⊂ ∆ contained in V ₀ .

Set ∆ 1 = ∆ − C 1 . Consider the commutative diagram

H

(M × M, M × M − ∆) ⊗ H

(M × (M, ∂M )) H

(M × M, M × M − ˙ ∆)

H

(M × M, M × M − ∆) ⊗ H

(M × (M, C)) H

(M × M, M × M − ∆

)

H

(V, V − ∆) ⊗ H

(V, V ∩ (M × C)) H

(V, V − ∆

)

H

(V, V − ∆) ⊗ H

(V, V

) H

(V, V − ∆

)

// //

²²

∼=

OO

∼=

²²

∼=

OO

//

²²

=

²² //

where the horizontal arrows are cup products, the vertical arrows are induced by the natural inclusions and the coefficients in any line are (R × Γ ) ^∗ ⊗ (R × R) → (R × Γ ) ^∗ . Consider the cup product U ∪ (1 × x) in the top line. In the bottom line we get U _|V ∪ (1 × x) _|V (here we de- note by U _|V the image of U under the natural homomorphism H ⁿ (M × M, M × M − ∆; (R × Γ ^∗ )) → H ⁿ (V, V − ∆; (R × Γ ^∗ )) and we identify x ∈ H ^p (M, ∂M ) = H ^p (M, C)). Considering the similar diagram for the cup product U ∪ (x × 1), in the bottom line we obtain U _|V ∪ (x × 1) _|V . But

(1 × x) _|V = (p ^∗ ₂ x) _|V = (p ^∗ ₁ x) _|V = (x × 1) _|V ∈ H ^p (V, V 0 )

since p _1|V , p _2|V : (V, V ₀ ) → (M, C) are homotopic. Since the right sides of both diagrams are identical and consist of isomorphisms,

(1 × x) _|V = (x × 1) _|V ∈ H ^n+p (V, V − ∆ 1 ) implies

U ∪ (1 × x) = U ∪ (x × 1) ∈ H ^n+p (M × M, M × M − ˙ ∆).

3. Poincar´ e duality. In this section we prove the Poincar´e Duality The- orem for a pair (K, L) of compact subspaces of a (non-orientable) manifold, where the Poincar´e map goes from ˇ Cech cohomology of a pair to homology by taking the cap product with the fundamental class. This is the form of the duality that we need in the proof of the Lefschetz coincidence theorem.

The Poincar´e Duality Theorem is also proved in [Sp2] (a Poincar´e map is defined by slant product and goes from homology to cohomology) and in [W1], [W2] (for any local coefficient system and some pairs of subspaces).

In our case we will modify Section VI of [B] for the non-orientable case.

Finally, we define the Thom class in the same fashion as in the orientable

case and consider the case of a manifold with boundary. Let us point out

that the Thom class can also be defined using duality as in [G1].

Let M be a topological n-manifold without boundary. In the rest of this paper Γ _M will denote the orientation system on M : Γ _M (x) = H _n (M, M − x; R). Then for any x ∈ M the module H _n (M, M − x; Γ _M ) has the canonical generator z M,x corresponding in

H _n (M, M − x; Γ _M ) = H _n (M, M − x; Z) ⊗ _Z H _n (M, M − x; R) to γ ⊗γ ⁰ where γ is a generator of H _n (M, M −x; Z) = Z and γ ⁰ is its image in H _n (M, M −x; R) under the map induced by the unique ring homomorphism Z → R.

Lemma 3.1 (fundamental class). For any compact set A ⊂ M there exists a unique element z _M,A ∈ H _n (M, M − A; Γ _M ) such that for any x ∈ A the natural homomorphism H n (M, M − A; Γ M ) → H n (M, M − x; Γ M ) sends z _M,A to z _M,x .

P r o o f. It is enough to check the conditions (i)–(iii) of the Bootstrap Lemma ([Bd; VI, 7.9]).

Now we can follow Chapter VI of [Bd] to reformulate the Poincar´e Du- ality Theorem for the non-orientable case.

For closed subsets L ⊂ K ⊂ M we denote by H ˇ ^p (K, L; G) = lim

−→ {H ^p (U, V ; G) : (U, V ) ⊃ (K, L), U, V open in M } the ˇ Cech cohomology modules. Let (K, L) ⊂ (U, V ) be as above. Then there is a cap product

∆ ^p (U, V ; G) ⊗ [(∆ n (V ; Γ ) + ∆ n (U − L; Γ ))/∆ n (U − K; Γ )]

→ ∆ ∩ _n−p (U − L, U − K; Γ ⊗ G) given by f ∩ (b + c) = f ∩ b + f ∩ c = f ∩ c. But

H _∗ ((∆ _∗ (V ; Γ ) + ∆ _∗ (U − L; Γ ))/∆ _∗ (U − K; Γ )) = H _∗ (U, U − K; Γ ⊗ G)

= H ∗ (M, M − K; Γ ⊗ G) since {V, U − L} is an open cover of U . Thus we get a cap product

H ^p (U, V ; G) ⊗ H n (M, M − K; Γ M ) → H n−p (M − L, M − K; Γ ⊗ G).

Thus capping with z _M,K ∈ H _n (M, M − K; Γ _M ) we get a homomorphism

∩z _M,K : H ^p (U, V ; G) → H _n−p (M − L, M − K; Γ ⊗ G)

which is compatible with the inclusion of (U, V ). Finally, in the direct limit we get the duality map

D _K,L : ˇ H ^p (K, L; G) → H _n−p (M − L, M − K; Γ ⊗ G).

Theorem 3.2 (Poincar´e duality). Let M be an n-manifold and let

K ⊃ L be compact subsets of M . Then the cap product map D K,L :

H ˇ ^p (K, L; G) → H _n−p (M − L, M − K; Γ ⊗ G) is an isomorphism.

P r o o f. Modify Lemmas 8.1, 8.2 and then follow the proof of Theorem 8.3 of Chapter VI in [Bd].

Now let M be a compact manifold with boundary. We will denote by Γ M the unique extension of the local system Γ int M onto M . Adding an open collar C to ∂M we get a manifold without boundary M ⁰ = M ∪ C. Applying Theorem 3.2 to M ⁰ with K = M and L = ∅ we get

Corollary 3.3. Let M be a compact manifold. Then the cap product

∩z _M,∅ : H ^p (M ; G) → H _n−p (M ⁰ , M ⁰ −M ; Γ _M

⊗G) = H _n−p (M, ∂M ; Γ _M ⊗G) is an isomorphism.

The element z M ∈ H n (M, ∂M ; Γ M ) corresponding to z _M,∅ ∈ H n (M ⁰ , M ⁰ − M ; Γ _M ⁰ ) (by the excision isomorphism) will be called the fundamental class of the manifold M with boundary. In the rest of this paper we put G = R. Then the isomorphism H ^p (M ; R) → H n−p (M, ∂M ; Γ M ) from Corol- lary 3.3 will be denoted by D _M (x) = x ∩ z _M .

Lemma 3.4 (Thom class; [Sp2; 4.7]). Let M be a manifold without bound- ary. There exists a unique element U _M ∈ H ⁿ (M × M, M × M − ∆;

(R × Γ M ) ^∗ ) whose restriction to any fibre is dual to the fundamental class, i.e. h(j _x ) ^∗ U _M , z _M,x i = 1 where j _x : (M, M − x) → (M × M, M × M − ∆) is given by j _x (y) = (x, y).

Let e U _M denote the image of U _M under the natural homomorphism k ^∗ : H ⁿ (M × M, M × M − ∆; (R × Γ _M ) ^∗ ) → H ⁿ (M × M ; (R × Γ _M ) ^∗ ).

Corollary 3.5. If M is closed then h e U _M , 1 × z _M i = 1.

P r o o f. Consider the commutative diagram

(M, ∅) (M × M, ∅)

(M, M − x) (M × M, M × M − ∆)

i ²²

l //

²² k k //

where l(y) = j(y) = (x, y), x ∈ M fixed. Now

h e U M , 1 × z M i = hk ^∗ (U M ), l ∗ (z M )i = hl ^∗ k ^∗ ( e U M ), z M i = h(kl) ^∗ ( e U M ), z M i

= h(ji) ^∗ ( e U _M ), z _M i = hj ^∗ ( e U _M ), i _∗ (z _M )i = hj ^∗ ( e U _M ), z _M,x i = 1.

Lemma 3.6. The inclusion (M × M − ∂∆, M × M − ∆) ⊂ (M × M, M × M − ∆) is a homotopy equivalence.

P r o o f. It is enough to find a deformation H : (M × M, M × M − ∆) ×

I → (M × M, M × M − ∆) satisfying H 0 = identity and H t (M × M ) ⊂

M × M − ∂∆, t ∈ (0, 1]. Then H ₁ : (M × M, M × M − ∆) ⊂ (M × M − ∂∆,

M × M − ∆) is a homotopy inverse to our inclusion. First we define a homotopy h : R ₊ × R ₊ × I → R ₊ × R ₊ by

h(t, s, τ ) =

 (t + (1 − t − s)τ /2, s + (1 − t − s)τ /2) for t + s ≤ 1,

(t, s) for t + s ≥ 1.

This homotopy shifts R + × R + into R + × R + − {(0, 0)}. Now we fix a collar ∂M × R ₊ in the manifold M , identify ∂M × R ₊ × ∂M × R ₊ =

∂M × ∂M × R + × R + and define a deformation of ∂M × ∂M × R + × R +

by H(x, y, t, s, τ ) = (x, y, h(t, s, τ max(0, 1 − d(x, y))). Since the carrier of this homotopy is locally compact, it extends (by identity) to the desired deformation of M × M .

Corollary 3.7. The inclusion (int M × int M, int M × int M − ∆) ⊂ (M × M, M × M − ∆) induces a (co-) homology group isomorphism.

P r o o f. The inclusion (int M × int M, int M × int M − ∆) ⊂ (M × M

− ∂∆, M × M − ∆) is excisive and (M × M − ∂∆, M × M − ∆) ⊂ (M × M, M × M − ∆) is a homotopy equivalence (Lemma 3.6).

Thus we get a commutative diagram of isomorphisms

H

(M × M, M × M − ∆; (R × Γ

)

)

H

(M × int M, M × int M − ∆; (R × Γ

)

) H

(int M × M, int M × M − ∆; (R × Γ

)

)

H

(int M × int M, int M × int M − ∆; (R × Γ

)

)

vv mmmm mmmm mmmm QQQQQQ QQQQQQ ((

QQQQQ QQQQQ Q(( vv mmm mmm mmm mm

(the upper arrows are induced by excision maps) and denote by U M

U _M ⁰⁰ U _M ⁰

U _{int M} zz uuu u IIII $$

IIII $$ zz uuu u

the elements corresponding to the Thom class U int M ∈ H ⁿ (int M × int M, int M × int M − ∆; (R × Γ _M ) ^∗ ) (Lemma 3.4). Let e U _M ⁰ , e U _M ⁰⁰ , e U _M denote the restrictions of these classes to H ⁿ (int M ×(M, ∂M )), H ⁿ ((M, ∂M )×int M ), H ⁿ ((M × M ), ∂(M × M ) − ∂∆) respectively.

Lemma 3.8. We have

h e U _M ⁰ , 1 × z _M i = 1

(here 1 ∈ H 0 (int M ; R) is the unit, and z M ∈ H n (M, ∂M ; Γ M ) is the fun-

damental class).

P r o o f. Consider the commutative diagram

(M, ∂M ) (int M × M, int M × ∂M )

(M, M − x) (int M × M, int M × M − ∆)

l //

i ²² _j ²² ^k

//

where l(y) = j(y) = (x, y), x ∈ int M fixed. Now

h e U _M ⁰ , 1 × z _M i = hk ^∗ (U _M ⁰ ), l _∗ (z _M )i = hU _M ⁰ , k _∗ l _∗ (z _M )i = hU _M ⁰ , j _∗ i _∗ (z _M )i

= hj ^∗ (U _M ⁰ ), z M,x i = hj ^∗ (U _{int M} ⁰ ), z int M,x i = 1, where the last equality follows from Lemma 3.4.

We will use the following notation: for a ∈ H ∗ M and x ∈ H ^∗ M we denote the corresponding elements by a ∈ H _∗ (int M ) and x ∈ H ^∗ (int M ).

Lemma 3.9. Let M be a compact n-manifold. Then for any x ∈ H ^p (M ; R) and a ∈ H p (M ; R),

h e U _M ⁰ , a × (x ∩ z _M )i = (−1) ^pn hx, ai.

P r o o f. We have

h e U _M ⁰ , a × (x ∩ z _M )i = h e U _M ⁰ , (1 ∩ a) × (x ∩ z _M )i (2.3)

= h e U _M ⁰ , (1 × x) ∩ (a × z _M )i (2.2)

= h e U int M ∪ (1 × x), a × z M i (2.4)

= h e U int M ∪ (x × 1), a × z M i

= h e U _M ⁰ ∪ (x × 1), a × z M i (2.2)

= h e U _M ⁰ , (x × 1) ∩ (a × z _M )i (2.3)

= (−1) ^pn h e U _M ⁰ , (x ∩ a) × (1 ∩ z _M )i

= (−1) ^pn hx, aih e U _M ⁰ , 1 × z _M i (3.8)

= (−1) ^pn hx, ai = (−1) ^pn hx, ai.

Lemma 3.10. Let M be a compact n-manifold. Then for any x ∈ H ^p (M,

∂M ; R) and a ∈ H p (M, ∂M ; R),

h e U _M ⁰⁰ , a × (x ∩ z _M )i = (−1) ^pn hx, ai,

where x∩z _M is regarded as an element of H _n−p (int M ; Γ _M ) = H _n−p (M ; Γ _M ).

P r o o f. We have

h e U _M ⁰⁰ , a × (x ∩ z M )i = h e U _M ⁰⁰ , (1 ∩ a) × (x ∩ z M )i (2.3)

= h e U _M ⁰⁰ , (1 × x) ∩ (a × z M )i

= h e U _M , (1 × x) ∩ (a × z _M )i (2.2)

= h e U _M ∪ (1 × x), a × z _M i (2.4)

= h e U M ∪ (x × 1), a × z M i (2.2)

= h e U M , (x × 1) ∩ (a × z M )i (2.3)

= (−1) ^pn hU _M , (x ∩ a) × (1 ∩ z _M )i

= (−1) ^pn hx, aih e U _M ⁰ , 1 × z M i (3.8)

= (−1) ^pn hx, ai.

A map g : M → N between two n-manifolds M, N is called orientation true if each α ∈ π ₁ M preserves a local orientation of M iff gα ∈ π ₁ N pre- serves a local orientation of N (see [O]). Notice that then any isomorphism of R-modules φ _x : Γ _M (x) → Γ _N (gx) (for fixed x) admits a unique extension to a morphism of local systems φ : Γ _M → Γ _N covering g. For R = Z (the integers) there are exactly two such morphisms. In the next sections we will associate with each orientation true map g a morphism φ obtained from one of the above two morphisms induced by the unique ring homomorphism Z → R.

4. The Lefschetz Theorem. In this section we give two extensions of the coincidence index and Lefschetz number onto pairs of maps f, g : (M, ∂M ) → (N, ∂N ) where M, N are compact n-manifolds and g is orienta- tion true. They generalize the classical cases: M , N closed oriented [V] and oriented [BS]. In the first approach we assume g(∂M ) ⊂ ∂N and we drop a similar assumption on f . Then we assume f (∂M ) ⊂ ∂N . The indices thus obtained may be different as the following simple example illustrates. Let f, g : (D ⁿ , S ⁿ⁻¹ ) → (D ⁿ , S ⁿ⁻¹ ) be maps of degree k, l respectively. If we require that only g preserves S ⁿ⁻¹ then f is homotopic to the constant map and we obtain the index (−1) ⁿ l. If we change the roles of f and g the index obtained equals k. We will show that in general the difference between these two indices is equal to the index of the restrictions f _| , g _| : ∂M → ∂N . In particular, these two indices are equal if M is closed.

First let us focus on the first generalization. We consider f, g : M → N

where g is orientation true and g(∂M ) ⊂ ∂N . Since f is homotopic to a

map into int N and any two such deformations are homotopic in int N , we

may assume that f (M ) ⊂ int N .

We define the index of the pair f, g as the image of the fundamental class z _M ∈ H _n (M, ∂M ; Γ _M ) under the sequence of homomorphisms

H _n (M, ∂M ; Γ _M ) → H ^d

_n (M × (M, ∂M ); R × Γ _M )

(f ×g)

−→ H n (int N × (N, ∂N ); R × Γ N ) ^{h e U} −→ R

^,·i and set ind ⁰ (f, g) = h e U _N ⁰ , (f × g) _∗ d ⁰ _∗ (z _M )i. This is an invariant with re- spect to homotopies f t , g t satisfying g t (∂M ) ⊂ ∂N . Moreover, ind ⁰ (f, g) 6= 0 implies f (x) = g(x) for an x ∈ M . Its sign depends on the choice of a morphism of local systems compatible with g.

We are going to define a suitable Lefschetz number.

Consider the homomorphisms

H ^q (N ; R) → H ^f

^q (M ; R) −→ H ^D

_n−q (M, ∂M ; Γ _M )

g

→ H _n−q (N, ∂N ; Γ _N ) ^D −→ H

^q (N ; R) and define θ ⁰ _q = D _N ⁻¹ g ∗ D M f ^∗ (q = 0, . . . , n).

From now on we assume that R is a field. We define the Lefschetz number of the pair f, g by

L ⁰ (f, g) = X n q=0

(−1) ^q tr θ ⁰ _q

where the right side denotes the alternating sum of the traces of the endo- morphisms θ _q of the finite-dimensional vector spaces H ^p (N ; R).

We are going to prove

Theorem 4.1 (normalization). Let f, g : M → N be a pair of maps be- tween compact n-manifolds with g orientation true and g(∂M ) ⊂ ∂N . Then

L ⁰ (f, g) = ind ⁰ (f, g).

In the proof we will follow [V] (Chapter 6).

Notice that in the definition of L ⁰ (f, g) we may consider the sequences H ^q (int N ; R) → H ^f

^q (M ; R) −→ H ^D

n−q (M, ∂M ; Γ M )

g

→ H _n−q (N, ∂N ; Γ _N ) ^D −→ H

^q (int N ; R) since we may assume that f (M ) ⊂ int N .

First we fix homogeneous bases of linear spaces over R:

a _i ∈ H _∗ (N ; R), a ⁰ _i ∈ H _∗ (N, ∂N ; Γ _N ), x _i ∈ H ^∗ (N ; R), x ⁰ _i ∈ H ^∗ (N, ∂N ; Γ _N ^∗ ) such that:

• the bases {a _i }, {x _i } are dual via the Kronecker index,

• the bases {a ⁰ _i }, {x ⁰ _i } are dual via the Kronecker index, and

• D _N (x _i ) = a ⁰ _i .

Similarly we fix bases

b i ∈ H ∗ (M ; R), b ⁰ _i ∈ H ∗ (M, ∂M ; Γ M ), y i ∈ H ^∗ (M ; R), y _i ⁰ ∈ H ^∗ (M, ∂M ; Γ _M ^∗ ) (here D _M (y _i ) = b ⁰ _i ). Suppose that the homomorphisms

f ^∗ : H ^∗ (int N ; R) → H ^∗ (M ; R), g ^∗ : H ^∗ (N, ∂N ; Γ _N ^∗ ) → H ^∗ (M, ∂M ; Γ _M ^∗ ) are given by f ^∗ (x _j ) = P

k γ _jk y _k and g ^∗ (x ⁰ _j ) = P

i β _ji y _i ⁰ . Then the dual homomorphisms

f ∗ : H ∗ (M ; R) → H ∗ (int N ; R), g ∗ : H ∗ (M, ∂M ; Γ M ) → H ∗ (N, ∂M ; Γ N ) are given by f ∗ (b i ) = P

k γ ki a k and g ∗ (b ⁰ _i ) = P

j β ji a ⁰ _j .

Lemma 4.2 (cf. [V; 6.10]). For fixed p the following equality holds in H n (int N × (N, ∂N ); R × Γ N ):

X

i

a i × g ∗ D M f ^∗ (x i ) = X

j

(f × g) ∗ (b j × b ⁰ _j )

where i runs over the set I _p = {i : dim a _i = p} and j runs over the set J p = {j : dim b j = p}.

P r o o f. It is enough to prove the similar equality X

i

a _i × g _∗ D _M f ^∗ (x _i ) = X

j

(f × g) _∗ (b _j × b ⁰ _j ) in H _n (N × (N, ∂N ); R × Γ _N ),

where f is considered as a map into N .

Consider g ∗ D M f ^∗ (x i ) ∈ H n−p (N, ∂N ; Γ N ). Since hx ⁰ _j , g _∗ D _M f ^∗ (x _i )i = hg ^∗ (x ⁰ _j ), D _M f ^∗ (x _i )i = D X

k

β _jk y ⁰ _k , X

l

γ _il b ⁰ _l E

= X

k,l

β _jk γ _il hy _k ⁰ , b ⁰ _l i = X

k

β _jk γ _ik ,

we get

g _∗ D _M f ^∗ (x _i ) = X

j,k

β _jk γ _ik a ⁰ _j .

Now X

i

a i × g ∗ D M f ^∗ (x i ) = X

i



a i ×  X

j,k

β jk γ ik a ⁰ _j



= X

i,j,k

β jk γ ik (a i × a ⁰ _j ).

On the other hand,

X

j

(f × g) ∗ (b j × b ⁰ _j ) = X

j

f ∗ (b j ) × g ∗ (b ⁰ _j )

= X

j

 X

i

γ _ij a _i



×  X

s

β _sj a ⁰ _s



= X

j,i,s

γ _ij β _sj (a _i × a ⁰ _s )), which implies our lemma.

Lemma 4.3 (cf. [V; 6.11]). Under the above notations, d ⁰ _∗ (z _M ) = X

j∈J

(−1) ^|b

^|·|b

^| b _j × b ⁰ _j

where |b| denotes the dimension of b ∈ H _∗ (M ; R) and J = S J _p . P r o o f. We rewrite the proof of (6.11) in [V]:

hy _k × y _l ⁰ , d ⁰ _∗ (z _M )i = hy _k ∪ y _l ⁰ , z _M i = (−1) ^|y

^|·|y

^| hy ⁰ _l ∪ y _k , z _M i

= (−1) ^|y

^|·|y

^| hy _l ⁰ , y k ∩ z M i = (−1) ^|b

^|·|b

^| hy ⁰ _l , b ⁰ _k i

= (−1) ^|b

^|·|b

^| δ _kl = (−1) ^|b

^|·|b

^| δ _kl . P r o o f o f T h e o r e m 4.1. Fix r = 0, . . . , n. Then

tr θ ⁰ _r = X

i∈I

hθ ⁰ _r (x i ), a i i

= X

i∈I

hD _N ⁻¹ g _∗ D _M f ^∗ (x _i ), a _i i (3.9)

= X

i∈I

(−1) ^nr h e U _N ⁰ , a _i × g _∗ D _M f ^∗ (x _i )i (4.2)

= X

j∈I

(−1) ^nr h e U _N ⁰ , (f × g) _∗ (b _j × b ⁰ _j )i.

Now L(f, g) =

X n r=0

(−1) ^r tr θ ⁰ _r (by the above equality)

= X n r=0

(−1) ^r X

j∈I

(−1) ^nr h e U _N ⁰ , (f × g) _∗ (b _j × b ⁰ _j )i

(since nr + r ≡ |b _j | · |b ⁰ _j | (mod 2) for j ∈ J _r )

=

D U e _N ⁰ , (f × g) _∗  X

j∈J

(−1) ^|b

^|·|b

^| b _j × b ⁰ _j

E

(4.3)

= h e U _N ⁰ , (f × g) _∗ (d ⁰ _∗ (z _M ))i = ind ⁰ (f, g).

Corollary 4.4. If f, g : M → N where g is orientation true and g(∂M ) ⊂ ∂N then L ⁰ (f, g) 6= 0 implies that f and g have a coincidence.

Now we are going to define an alternative index and Lefschetz number.

We consider a pair f, g : M → N where f (∂M ) ⊂ ∂N and g is orientation true. The definitions will be similar to the previous ones but the symmetry is not full (g is orientation true in both cases) hence we will sketch the proof of the second version of the Lefschetz theorem.

Consider a pair f, g : M → N where f (∂M ) ⊂ ∂N and g is orientation true. We define the coincidence index of this pair as the image of the funda- mental class z M ∈ H n (M, ∂M ; Γ M ) under the sequence of homomorphisms

H _n (M, ∂M ; Γ _M ) −→ H ^d

_n ((M, ∂M ) × int M ; R × Γ _M )

(f ×g)

−→ H n ((N, ∂N ) × int N ; R × Γ N ) ^{h e U} −→ R

^,·i and set ind ⁰⁰ (f, g) = h e U _N ⁰⁰ , (f × g) ∗ d ⁰⁰ _∗ (z M )i. This is an invariant with respect to homotopies f _t , g _t satisfying f _t (∂M ) ⊂ ∂N . Moreover, ind ⁰⁰ (f, g) 6= 0 implies f (x) = g(x) for an x ∈ M .

Now we define the corresponding Lefschetz number. We consider the homomorphisms

H ^q (N, ∂N ; R) → H ^f

^q (M, ∂M ; R) −→ H ^D

_n−q (M ; Γ _M )

g

→ H _n−q (N ; Γ _N ) ^D −→ H

^q (N, ∂N ; R) and set θ _q ⁰⁰ = D _N ⁻¹ g _∗ D _M f ^∗ (q = 0, . . . , n).

We define the second Lefschetz number as L ⁰⁰ (f, g) =

X n q=0

(−1) ^q tr θ _q ⁰⁰ .

Theorem 4.5 (normalization). Let f, g : M → N be a pair of maps between compact n-manifolds with g orientation true and f (∂M ) ⊂ ∂N . Then

L ⁰⁰ (f, g) = ind ⁰⁰ (f, g).

Notice that in the definition of L ⁰⁰ (f, g) we may consider the sequences H ^q (N, ∂N ; R) → H ^f

^q (M, ∂M ; R) −→ H ^D

_n−q (int M ; Γ _M )

g

→ H _n−q (int N ; Γ _N ) ^D −→ H

^q (N, ∂N ; R)

since we may assume that g(M ) ⊂ int N .

First we fix homogeneous bases of linear spaces over R:

a i ∈ H ∗ (N, ∂N ; R), a ⁰ _i ∈ H ∗ (N ; Γ N ), x i ∈ H ^∗ (N, ∂N ; R), x ⁰ _i ∈ H ^∗ (N ; Γ _N ^∗ ) such that:

• the bases {a _i }, {x _i } are Kronecker dual,

• the bases {a ⁰ _i }, {x ⁰ _i } are Kronecker dual, and

• D _N (x _i ) = a ⁰ _i . Similarly we fix bases

b i ∈ H ∗ (M, ∂M ; R), b ⁰ _i ∈ H ∗ (M ; Γ M ), y i ∈ H ^∗ (M, ∂M ; R), y ⁰ _i ∈ H ^∗ (M ; Γ _M ^∗ ) (here D M (y i ) = b ⁰ _i ).

Suppose that the homomorphisms

f ^∗ : H ^∗ (N, ∂N ; R) → H ^∗ (M, ∂M ; R), g ^∗ : H ^∗ (N ; Γ _N ^∗ ) → H ^∗ (M ; Γ _M ^∗ ) are given by f ^∗ (x _j ) = P

k γ _jk y _k and g ^∗ (x ⁰ _j ) = P

i β _ji y _i ⁰ . Then the dual homomorphisms

f _∗ : H _∗ (M, ∂M ; R) → H _∗ (N, ∂N ; R), g _∗ : H _∗ (M ; Γ _M ) → H _∗ (N ; Γ _N ) are given by f _∗ (b _i ) = P

k γ _ki a _k and g _∗ (b ⁰ _i ) = P

j β _ji a ⁰ _j .

Lemma 4.6 (cf. [V; 6.10]). For fixed p the following equality holds in H _n ((N, ∂N ) × N ; R × Γ _N ):

X

i

a _i × g _∗ D _M f ^∗ (x _i ) = X

j

(f × g) _∗ (b _j × b ⁰ _j )

where i runs over the set I p = {i : dim a i = p} and j runs over the set J _p = {j : dim b _j = p}.

Lemma 4.7 (cf. [V; 6.11]). Under the above notations, d ⁰ _∗ (z M ) = P

j∈J (−1) ^|b

^|·|b

^| b _j × b ⁰ _j where |b| denotes the dimension of b ∈ H _∗ (M ; R).

P r o o f o f T h e o r e m 4.5. Fix r = 0, . . . , n. Then tr θ ⁰⁰ _r = X

i∈I

hθ _r ⁰⁰ (x _i ), a _i i

= X

i∈I

hD _N ⁻¹ g _∗ D _M f ^∗ (x _i ), a _i i (3.10)

= X

i∈I

(−1) ^nr h e U _N ⁰⁰ , a _i × g _∗ D _M f ^∗ (x _i )i (4.6)

= X

j∈J

(−1) ^nr h e U _N ⁰⁰ , (f × g) ∗ (b j × b ⁰ _j )i.

Now L ⁰⁰ (f, g) =

X n r=0

(−1) ^r tr θ ⁰⁰ _r (by the above equality)

= X n r=0

(−1) ^r X

j∈J

(−1) ^nr h e U _N ⁰⁰ , (f × g) _∗ (b _j × b ⁰ _j )i

(since nr + r ≡ |b _j | · |b ⁰ _j | (mod 2) for j ∈ J _r )

=

D U e _N ⁰⁰ , (f × g) _∗  X

j∈J

(−1) ^|b

^|·|b

^| b _j × b ⁰ _j

E

(4.7)

= h e U _N ⁰⁰ , (f × g) ∗ (d ⁰⁰ _∗ (z M ))i = ind ⁰⁰ (f, g)i.

Corollary 4.8. If f, g : M → N where g is orientation true and f (∂M ) ⊂ ∂N then L ⁰⁰ (f, g) 6= 0 implies that f, g have a coincidence.

Now we are going to show a theorem connecting the above two Lefschetz numbers. We notice (cf. [Bd; VI.9.1]) that the image of the fundamental class z _M ∈ H _n (M, ∂M ; Γ _M ) under the connecting homomorphism of the pair (M, ∂M ) is the fundamental class of the boundary: ∂ ∗ (z M ) = z ∂M ∈ H _n−1 (∂M ; Γ _M ).

Theorem 4.9. Let f, g : (M, ∂M ) → (N, ∂N ), g orientation true. Then L ⁰ (f, g) − L ⁰⁰ (f, g) = L(∂f, ∂g).

P r o o f. Consider the commutative (up to sign) diagram

H

(∂N ; R) H

(N, ∂N ; R) H

(N ; R) H

(∂N ; R)

H

(∂M ; R) H

(M, ∂M ; R) H

(M ; R) H

(∂M ; R)

H

(∂M ; Γ

) H

(M ; Γ

) H

(M, ∂M ; Γ

) H

(∂M ; Γ

)

H

(∂N ; Γ

) H

(N ; Γ

) H

(N, ∂N ; Γ

) H

(∂N ; Γ

)

H

(∂N ; R) H

(N, ∂N ; R) H

(N ; R) H

(∂N ; R)

// //

f^∗

²²

(−1)

//

f^∗

²² //

f^∗

²²

(−1)

f^∗

²² // //

D_∂M

²² //

DM

²² //

DM

²²

D_∂M

²² // //

g^∗

²²

(−1)

//

g^∗

²² //

g^∗

²²

(−1)

g^∗

²² // //

D_∂N⁻¹

²² //

D⁻¹_N

²² //

D⁻¹_N

²²

D⁻¹_∂N

²² // // // //

Composing the vertical arrows we get a self-map of the long exact se-

quence

H ^q−1 (∂N ; R) H ^q (N, ∂N ; R) H ^q (N ; R) H ^q (∂N ; R)

H ^q−1 (∂N ; R) H ^q (N, ∂N ; R) H ^q (N ; R) H ^q (∂N ; R)

// //

θ

²² //

θ

²² //

θ

²²

θ

²² //

// // // // //

which implies P _∞

q=0 (−1) ^q [tr θ ⁰⁰ _q − tr θ ⁰ _q + tr θ _q ^∂ ] = 0, hence X ∞

q=0

(−1) ^q tr θ ⁰⁰ _q − X ∞ q=0

(−1) ^q tr θ _q ⁰ + X ∞ q=0

(−1) ^q tr θ ^∂ _q = 0

and finally

L ⁰⁰ (f, g) − L ⁰ (f, g) + L(∂f, ∂g) = 0.

The Lefschetz coincidence number for manifolds with boundary in the oriented case was introduced in [M]. Notice that Λ(f, g) defined in that paper is equal to our (−1) ⁿ L ⁰⁰ (g, f ).

The above theorem implies that in the closed case both Lefschetz numbers L ⁰ (f, g) and L ⁰⁰ (f, g) are equal and it would be natural to expect that in this case this number satisfies L(f, g) = (−1) ⁿ L(g, f ) (as in the oriented case). We will show that this equality is true if g is the identity map. However, we will give examples showing that it is not true in general.

Definition 4.10. Let f : M → M be an orientation true map be- tween closed n-manifolds. We define the degree of f as the natural number k satisfying f ∗ (z M ) = k · z N .

We denote it by deg(f ). The sign of this number depends on the choice of a morphism of the local systems. However, in the case of coverings there is a natural morphism.

Lemma 4.11. If p : f M → M is a k-fold covering then deg(p) = k.

P r o o f. Consider the commutative diagram H _n ( f M ; Γ _{M f} ) H _n ( f M , f M − p ⁻¹ (x); Γ _{M f} ) = L

H _n ( e U _α , e U _α − e x _α ; Γ _{M f} )

H _n (M ; Γ _M ) H _n (M, M − x; Γ _M ) = H _n (U, U − x; Γ _U )

//

²² ²² ²²

∼ = //

and notice that each fundamental class z α ∈ H n ( e U α , e U α − e x α ; Γ _{M f} ) is sent

by the rightmost vertical arrow to z _U,x .

Corollary 4.12. If p M : f M → M and p N : e N → N are k-fold cover- ings and the diagram

H n ( f M ; Γ _{M f} ) H _n ( e N ; Γ _{N f} )

H _n (M ; Γ _M ) H _n (N ; Γ _N )

f e

//

p

²²

p

²² f //

commutes then deg(f ) = deg( e f ).

P r o o f. Indeed,

k · deg(f ) = deg(p M ) · deg(f ) = deg(f p M ) = deg(p N f ) e

= deg(p _N ) · deg( e f ) = k · deg( e f ).

Example 4.13. Let e f _k : S ² → S ² be an odd map ( e f _k (−x) = − e f _k (x)) of degree k (k must then be an odd number). This map induces a map f k : RP ² → RP ² . By the above corollary deg(f k ) = deg( e f k ) = k. On the other hand, f _k induces an isomorphism of fundamental groups and hence is orientation true. Let us fix two odd integers k, l and compute the Lefschetz number L(f k , f l ) (with rational coefficients R = Q). Since RP ² is Q-acyclic, only the sequence

H ⁰ (RP ² ; Q) ^f

∗

→ H

⁰ (RP ² ; Q) → H 2 (RP ² ; Γ ) ^f

∗

→ H

2 (RP ² ; Γ ) → H ⁰ (RP ² ; Q) may have a non-zero contribution to L(f _k , f _l ). Thus L(f _k , f _l ) = deg(f _l ) = l.

But on the other hand, L(f _l , f _k ) = deg(f _k ) = k.

Example 4.14. The following example shows that L(f, g) 6= 0 does not imply L(g, f ) 6= 0. Let p, c : S ² → RP ² where p is the projection and c is a constant map. It is easy to see that L(p, c) = 0 but L(c, p) = 2.

Despite the above examples, we do have the following positive result:

Proposition 4.15. Let f : M → M be an orientation true map of a closed n-manifold. Then L(id, f ) = (−1) ⁿ L(f, id).

P r o o f. Let us denote by (f _q , R), (f ^q , R), (f _q , Γ _M ) the self-homomor- phisms induced by f on H q (M ; R), H ^q (M ; R) and H q (M ; Γ M ) respectively.

Then certainly tr(f ^q , R) = tr(f _q , R).

By the definition given at the beginning of this section, L(f, id) =

X ∞ q=0

(−1) ^q tr(f ^q , R), L(id, f ) = X ∞ q=0

(−1) ^q tr(f _n−q , Γ _M ).

Since (−1) ^q = (−1) ⁿ (−1) ^n−q , it remains to show that P _∞

q=0 (−1) ^q tr(f _q , R)

= P _∞

q=0 (−1) ^q tr(f q , Γ M ).

Let K be a cell structure of M . Recall that H ∗ (M ; Γ ) (Γ = R or Γ M ) may be obtained from the chain complex (C _i , ∂ _i ) where C _i = H _i (K ⁱ , K ⁱ⁻¹ ; Γ ) with suitable boundary operators ∂ _i [Wh]. These boundary operators may be different in the two cases considered but certainly, by excision, H _i (K ⁱ , K ⁱ⁻¹ ; Γ ) does not depend on Γ . Let f _i : C _i → C _i be the in- duced maps at the chain level. Now we recall that the Lefschetz num- ber of a chain map and of the induced homology map are equal. Thus P _∞

i=0 (−1) ⁱ tr(f _i ) = P _∞

i=0 (−1) ⁱ tr(f _i , Γ ). Since the left hand side of this equality does not depend on the boundary operator, the right hand sides are equal in the two cases: Γ = R and Γ = Γ _M , giving the desired equality.

R e m a r k s. 1) More explicit computations of L(f, g) have been done for maps f , g between compact surfaces with g orientation true in [GO].

These computations rely basically on Lemma 4.3 or more precisely on its dual which is Proposition 2.3 of [G1].

2) It is no surprise that the numbers L(f, g) and L(g, f ) may be differ- ent. Notice that in our definition of L(f, g) the homology homomorphism induced by the map (f, g) has values in the group H _n (M × M ; R × Γ _M ) where the coefficients are not symmetric in the two coordinates. In fact, L(f, g) and L(g, f ) are related to two different geometric situations. More specifically, the Lefschetz coincidence number is the sum of the indices of all coincidence points (we may assume that their number is finite). To com- pute these indices, we first fix a local orientation at a coincidence base point x ₀ ∈ M and a local orientation at y ₀ = f (x ₀ ) = g(x ₀ ) ∈ N (compare the definition of semi-index in the next section). Let x ₁ be another coincidence point. We join these two points with a path γ and we translate the local orientation from x ₀ to x ₁ along γ. To get the contribution of the coincidence point x 1 to L(f, g) and L(g, f ) we translate the local orientation from y 0

to y ₁ = f (x ₁ ) = g(x ₁ ) along the path g(γ) or f (γ) respectively. If x ₀ and x ₁ do not belong to the same Nielsen class then these paths may not be homotopic and may give different local orientations.

5. Coincidence index and semi-index. In this section we will define a local coincidence index for a pair of maps f, g : M → N where M, N are (possibly non-compact) n-manifolds without boundary, the coincidence set is compact, and the map g is orientation true. We will also consider homo- topies F, G : M × I → N with Φ(F, G) = {(x, t) ∈ M × I : F (x, t) = G(x, t)}

compact. We will call them Φ-compact pairs of maps and Φ-compact pairs

of homotopies respectively. For any clopen subset C ⊂ Φ(f, g) we will de-

fine the coincidence index generalizing the index from the previous section

(for M , N closed, C = Φ(f, g)). We will show that for any Nielsen

class the absolute value of this index equals the coincidence semi-index

from [DJ], [Je].