III.Ultrahyperbolicequationson R AnalysisontheminimalrepresentationofO ð p ; q Þ ARTICLEINPRESS

http://www.elsevier.com/locate/aim Advances in Mathematics 180 (2003) 551–595

Analysis on the minimal representation of Oðp; qÞ III.Ultrahyperbolic equations on R ^{p 1;q 1}

Toshiyuki Kobayashi

and Bent Ørsted

aRIMS, Kyoto University, Sakyo-ku, Kyoto 606-8502, Japan

bDepartment of Mathematics and Computer Science, SDU-Odense University, Campusvej 55, Odense M, DK-5230, Denmark

Received 5 November 2001; accepted 3 September 2002 Communicated by Bertram Kostant

Abstract

For the group Oðp;qÞwe give a new construction of its minimal unitary representation via Euclidean Fourier analysis.This is an extension of theq¼2 case, where the representation is the mass zero, spin zero representation realized in a Hilbert space of solutions to the wave equation.The group Oðp;qÞ acts as the Mo¨bius group of conformal transformations on R^p1;q1; and preserves a space of solutions of the ultrahyperbolic Laplace equation on R^p1;q1:We construct in an intrinsic and natural way a Hilbert space of solutions so that Oðp;qÞbecomes a continuous irreducible unitary representation in this Hilbert space.We also prove that this representation is unitarily equivalent to the representation onL²ðCÞ;whereC is the conical subvariety of the nilradical of a maximal parabolic subalgebra obtained by intersecting with the minimal nilpotent orbit in the Lie algebra of Oðp;qÞ:

r2003 Published by Elsevier Science (USA).

MSC:22E45; 22E46; 35C15; 35L82

Keywords:Minimal unitary representation; Ultrahyperbolic equation

Contents

1. Introduction ... 552 2.Ultrahyperbolic equation onR^p1;q1and conformal group ... 558

Corresponding author.

E-mail addresses:toshi@kurims.kyoto-u.ac.jp (T. Kobayashi), orsted@imada.sdu.dk (B. Ørsted).

0001-8708/$ - see front matterr2003 Published by Elsevier Science (USA).

doi:10.1016/S0001-8708(03)00014-8

1. Introduction

1.1. In this paper, we study the symmetries of the ultrahyperbolic Laplace operator on a real finite-dimensional vector space equipped with a non-degenerate symmetric bilinear form.We shall work in coordinates so that the operator becomes

&_Rp1;q1&_z:¼ @²

@z²₁þ?þ @²

@z²_p1 @²

@z²_p? @²

@z²_pþq2;

onRⁿ¼R^p1;q1:In the case of Minkowski space (q¼2) we are studying the wave equation, which is well-known to have a conformally invariant space of solutions, see [14].This corresponds to the fact that the equation &_Rp1;q1f ¼0 in this case describes a particle of zero mass.Incidentally, it may also be interpreted as the bound states of the Hydrogen atom, namely each energy level corresponds to a K-type—for ðp;qÞ ¼ ð4;2Þ: This gives the angular momentum values by further restriction to Oð3Þ:In general the indefinite orthogonal groupG¼Oðp;qÞacts as the Mo¨bius group of meromorphic conformal transformations on R^p1;q1; leaving a space of solutions to&_Rp1;q1f ¼0 invariant.

1.2. The main object of the present paper is to construct in an intrinsic and natural way a Hilbert space of solutions of&_Rp1;q1 so that the action of Oðp;qÞbecomes a continuousunitaryirreducible representation in this Hilbert space forðp;qÞsuch that p;qX2 andpþq44 is even.From an algebraic view point of representation theory, our representations are:

(i) minimal representations ifpþqX8 (i.e. the annihilator is the Joseph ideal).

(ii) notspherical ifpaq (i.e. no non-zeroK-fixed vector).

(iii) nothighest weight modules of SO0ðp;qÞifp;qX3:

In a long history of representation theory of semisimple Lie groups, it is only quite recent that our representations for p;qX3 have been paid attention, especially as minimal unitary representations; they were first discovered by Kostant [13] for ðp;qÞ ¼ ð4;4Þ and generalized by Binegar–Zierau [3] as subrepresentations of degenerate principal series representations.There is also another algebraic approach to the same representations by using the theta correspondence for the trivial

3. Square integrable functions on the cone ... 569 4. Green function and inner product ... 572 5. Bessel function and an integral formula of spherical functions .... 582 6.Explicit inner product on solutions&_Rp1;q1f ¼0 . . . 589

representation of SLð2;RÞ by Huang–Zhu.Our previous papers [11,12] treated the same representation by geometric methods and with other points of view.We think that such various approaches reflect a rich structure of the minimal representations.

It is perhaps of independent interest that the (in some sense maximal group of) symmetries Oðp;qÞof the space of solutions of&_Rp1;q1f ¼0 lead to such a natural Hilbert space.Our inner product ð ; Þ_W defined by an integration over a non- characteristic hyperplane (see (1.5.1)) is a generalization of the one coming from energy considerations in the case of wave equations, and even the translation invariance of the inner product contains some new information about solutions.

It is also of independent interest from the representation theory of semisimple Lie groups that our representations are unitarily equivalent to the representations on L²ðCÞ; where C is the null cone of the quadratic form on R^p1;q1: This result is proved via the Fourier transform in Theorem 4.9. SuchL²-realizations of ‘‘unipotent representations’’ is expected from the philosophy of the Kostant–Kirillov orbit method, but has not been proved except for some special cases of highest weight modules or spherical representations.

We have avoided most of the references to the theory of semisimple Lie groups and representation theory, and instead given direct constructions of the key objects, such as for example the minimalK-type; this is given as an explicit hyper- geometric function, and we also calculate its Fourier transform in terms of a Bessel function.By application of explicit differential operators forming the Lie algebra of Gwe can generate the whole Hilbert space of solutions beginning from the minimal K-type.

1.3. For q¼2 (or p¼2) we are dealing with highest weight representations (when restricting to the identity component SO₀ðp;2Þ), and these have been studied by many authors, in particular in the physics literature.For a nice introduction to this representation and its construction via geometric quantization (and more), see[8].In this case the K-types may be identified with energy levels of the bound states of the Hydrogen atom, and the smallest one with the bound state of lowest energy.

We can summarize the situation, covering both the classical Kepler problem and its quantization in case q¼2; as in the diagram below.Here the left-hand side represents the classical descriptions of, respectively, the Kepler problem and geodesic flow on the sphere; by ‘‘symplectic transform’’ we are alluding to the change of variables between these two Hamiltonian systems as presented in[8].

The right-hand side involves the quantizations of these two systems, where the wave-equation is considered as the quantization of geodesic flow, also to be thought of as geometric optics.The quantum analogue of the ‘‘symplectic transform’’ involves the Fourier transform.Finally, we invoke conformal geometry and combine it with the Fourier transform, which in a different (and new as far as constructing Hilbert spaces and unitary actions) way appears in passing from the wave equation to the Fourier realization of solutions—this is the last arrow on

the right-hand side.

The main focus of this paper is on boxes (A) and (B).In particular, we give an explicit inner product in model (A) (Theorems 1.4 and 1.5) and construct via Fourier transform a new realization of the minimal representation (Theorem 1.6) for general p;q:

1.4. From now, suppose that n:¼pþq2 is an even integer greater than 2; and p;qX2:Let us briefly state some of our main results in a more explicit way.

First, we find a formula of Green’s function E₀ for the ultrahyperbolic Laplace operator&_Rp1;q1;in Proposition 4.2, namely,E₀ is given by a constant multiple of the imaginary part of the regularized Schwartz distribution:

e ffiffiffiffiffi

p1 pðq1Þ

2 ðx²₁þ?þx²_p1x²_p?x²_pþq1þ ffiffiffiffiffiffiffi p1

0Þ¹ⁿ²:

See also the recent paper of Ho¨rmander [9] for further details on distributions associated with this ultrahyperbolic equation.Then we construct solutions of

&_Rp1;q1f ¼0 by the integral transformation:

S:C₀^NðRⁿÞ-C^NðRⁿÞ; j/E₀j ðseeð4:3:1ÞÞ:

The image SðC^N₀ ðRⁿÞÞ turns out to be ‘‘large’’ in Ker&_Rp1;q1 (see Section 4.7, Remark (2)).On this image, we define a Hermitian formð; Þ_N by

ðf1;f2Þ_N :¼ Z

Rⁿ

Z

Rⁿ

E0ðyxÞj₁ðxÞj₂ðyÞdx dy; ð1:4:1Þ wheref_i¼E₀j_i ði¼1;2Þ:Here is a part of Theorem 4.7, which is the first of our main results:

Theorem 1.4. ð ; Þ_Nis positive-definite on the image of S:Furthermore, Oðp;qÞacts as an irreducible unitary representation on its Hilbert completionH:

We shall writeð$^min_Rp1;q1;HÞ for this unitary representation.We also prove that this representation is isomorphic to the minimal representation of Oðp;qÞ;

constructed previously by Kostant, Binegar–Zierau[3,13]and also in our previous papers[11,12] from different viewpoints.Thus, Theorem 1.4 may be regarded as a realization of the minimal representation (with an explicit inner product) in the solution space of the ultrahyperbolic equation.

1.5. The above definition of the inner productð ; Þ_N (see (1.4.1)) uses the integral expression of solutions of &

R^p1;q1f ¼0: Can we write the inner product without

knowing the preimage? Yes, the second of our main results is to give an intrinsic inner product on the same solution space by using the Cauchy data.For simplicity, we takez₁¼0 as a non-characteristic hyperplane.Then, we decompose a solution

f ¼fþþf

such that f₇ðz1;y;z_nÞ is holomorphic with respect to the first variable z₁ in the complex domain offz1AC:7Imz₁40gof thez₁-variable.This is an expression off as a hyperfunction, and such a pairðfþ;fÞcan be obtained by the convolution in the z₁-variable (see (6.2.3)):

f₇ðzÞ ¼ 1 2p ffiffiffiffiffiffiffi

p1 81 z₁7 ffiffiffiffiffiffiffi

p1

0fðz1;y;z_nÞ;

where the integration makes sense for f with suitable decay at infinity.Then we define a Hermitian form

ðf;fÞ_W :¼ 1 ffiffiffiffiffiffiffi p1

Z

Rⁿ¹

fþ

@f_þ

@z1

f

@f

@z1

_z

1¼0

dz2?dzn: ð1:5:1Þ Then we shall prove that ð ; Þ_W is independent of the specific choice of a non- characteristic hyperplane, as follows from the (non-trivial) isometric invariance.

Much more strongly,ð ; Þ_W is conformally invariant.A precise formulation for this is given in Theorem 6.2, which includes:

Theorem 1.5. 4pð ; Þ_W ¼ ð ; Þ_N:In particular,ð ; Þ_W is positive definite andOðp;qÞ- invariant.

Hence, in place of Theorem 1.4, we can obtain the same irreducible unitary representation of Oðp;qÞ on the Hilbert completion of a space of solutions with respect to the inner productð ; Þ_W:

An interesting property of this inner product is its large invariance group.Even in the case of the usual wave equation (q¼2 case) our approach to the Hilbert space of solutions and the corresponding representation offers some new points of view.In this case, if we take the non-characteristic hyperplane as fixed time coordinate (namely, ifz1 stands for the time), then the translational invariance amounts to a

remarkable ‘‘conservation law’’.Instead, we can take the non-characteristic hyperplane by fixing one of the space coordinates, and an analogous integration over the hypersurface (containing the time coordinate) still gives the same inner product ! As a final remark in Section 6.7, we note the connection to the theory of conserved quantities for the wave equation (q¼2 case), such as the energy and others obtained by the action of the conformal group.

1.6. The Gelfand–Kirillov dimension of our representationð$^min

R^p1;q1;HÞispþq

3: So, we may expect that the representation could be realized on a ðpþq3Þ- dimensional manifold.For this purpose, we define the null cone of the metric as

C:¼ fzARⁿ:z²₁þ?þz²_p1z²_p?z²_n¼0g:

The third of our main results is another realization of the unitary representation

ð$^min_Rp1;q1;HÞ in a function space on a ðpþq3Þ-dimensional manifold C: The

Fourier transformFmaps solutions of&

R^p1;q1f ¼0 to distributions supported on

the null coneC:Surprisingly, the inner product of our Hilbert space turns out to be simply theL²-norm onCwith respect to a canonical measuredm(see (3.3.3)). Here is a part of Theorem 4.9: We regardL²ðCÞas a subspace of distributions by a natural injective mapT:L²ðCÞ-S⁰ðRⁿÞ:

Theorem 1.6. ð2pÞⁿ²T¹3Fis a surjective unitary operator fromH to L²ðCÞ:

Theorem 1.6 defines an irreducible unitary representation of G¼Oðp;qÞ on L²ðCÞ; denoted by p; which is unitarily equivalent to ð$^min_Rp1;q1;HÞ: Since the maximal parabolic subgroupP^max of G (see Section 2.7) acts onR^p1;q1 as affine transformations, the restriction pj_Pmax has a very simple form, namely, the one obtained by the classical Mackey theory (see (3.3.5)). In this sense, Theorem 1.6 may be also regarded as an extension theorem of an irreducible unitary representation from the maximal parabolic subgroupP^maxto the whole groupG:

1.7. The fourth of our main results is about the representationðp;L²ðCÞÞas aðg;KÞ- module on the Fourier transform side, especially to find an explicit vector in the minimalK-type.

In the realization onL²ðCÞ; the actionpðgÞis not simple to describe except for gAP^max: Instead, we consider the differential action dp of the Lie algebra g₀ on smooth vectors ofL²ðCÞ; which turns out to be given by differential operators at most of second order (see Section 3.2). This makes the analogy with the metaplectic representation (whereGis replaced by the symplectic group) a good one.Here we are recalling the fact, that the even part of the metaplectic representation may be realized as anL²-space of functions on the cone generated by rank one projections inRⁿ:

Moreover, by using a reduction formula of an Appell hypergeometric function, we find explicitly the Fourier transform of a Jacobi function multiplied by some conformal factor which equals to a scalar multiple of

c_0;eðzÞ:¼ jzj^3q2 Kq3 2

ð2jzjÞdmAS⁰ðRⁿÞ:

Here KnðzÞ is a modified Bessel function of the second kind.This vector c_0;eðzÞ corresponds to the bound state of lowest energy forq¼2 case.For generalp;q;the K-span ofc_0;eðzÞgenerates the minimalK-type in the realization onL²ðCÞ:

We define a subspaceUofS⁰ðRⁿÞto be the linear span of its iterative differentials dpðX1Þ?dpðXkÞc_0;eðzÞ ðX1;y;X_kAg₀#RCÞ:

What comes out of Section 5 may be formulated in this way (combining with Theorem 4.9, see Section 3.2 for notation): SupposepþqA2Z;pþq44 andpXqX2:

Theorem 1.7. (1)jzj

3q 2 Kq3

2

ð2jzjÞis a K-finite vector in L²ðCÞ:

(2) U is an infinitesimally unitaryðg;KÞ-module via$#n2 2 ;e: (3) U is dense in the Hilbert space TðL²ðCÞÞ:

(4) The completion of (2) defines an irreducible unitary representation of G on TðL²ðCÞÞ;and then also on L²ðCÞ:

In the paper [4] one finds a similar construction of Hilbert spaces and unitary representations for Koecher–Tits groups associated with semisimple Jordan algebras under the assumption that the representations are spherical, and there also occur Bessel functions as spherical vectors.In our situation the representations are not spherical if paq: Our approach is completely different from [4] that treats some spherical representations, and contrary to what is stated in[4, p.206]we show that forG¼Oðp;qÞ(pþqeven) it is possible to extend the Mackey representation of the maximal parabolic subgroup to the whole group.Furthermore, even forp¼qcase, our approach to Theorem 1.7 has an advantage that we give the exact constants normalizing the unitary correspondence between the minimal K-type in other realizations and the Bessel function in our realization onL²ðCÞ(see Theorem 5.5).

1.8. The paper is organized as follows: We begin by recalling some results from conformal geometry and facts about the conformal group, in particular in[11].In Section 3 we give the basic setup for a realization on the null cone via Fourier transform.Then we construct the intertwining operator from the minimal representation to the model treated here and calculate the new expression for the inner product (see Theorem 1.4). We show in Proposition 4.2 that the Green function

of&_Rp1;q1has a Fourier transform equal to the invariant measure on the null-cone,

allowing one more expression for the inner product (see Theorem 1.6); also we

obtain from this an intertwining operator from test functions to solutions.Indeed, in Section 4, Proposition 4.6 we prove that the Green function is up to a constant exactly the kernel in the Knapp–Stein intertwining integral operator between degenerate principal series representations at the parameters we study; this enables us to understand the unitarity of the minimal representation on the modelR^p1;q1in an elementary and explicit way.Note that all normalizing constants are computed explicitly.Lemma 2.6 states the irreducibility and unitarizability, which we use; we give in[12], Sections 7.6 and 8.3 independent proofs of these facts.

In Section 5 we construct the lowestK-type as a modified Bessel function, whose concrete properties are important forK-type information aboutL²ðCÞ:The idea here is to use a classical formula on the Hankel transform due to Baily in 1930s, and then apply reduction formulae of an Appell hypergeometric function of two variables.

Section 6 contains formulae for the inner productð; Þ_W in terms of integration over a Cauchy hypersurface.Summarizing, we give five different realizations of the inner product together with the normalizations of these relative to each other.

Namely, in addition toð ; Þ_N andð ; Þ_W we also define three more:ð; Þ_M (coming from a pseudo-differential operator on M¼S^p1S^q1), ð; Þ_A (coming from a normalized Knapp–Stein intertwining operator), and finally ð ; Þ_C; which is just L²ðCÞ:This is seen in the key diagram (see Section 4.11)

C^N₀ ðRⁿÞ !^S C_n2

2

ðD*_MÞ !^FS⁰ðRⁿÞ%^T L²ðCÞ; -

Ker&_Rp1;q1;

where the spaces correspond to four different ways of generating solutions to our ultrahyperbolic equation.S will be an integral transform against the Green kernel (essentially, a Knapp–Stein intertwining operator with a specific parameter), andF the Fourier transform, mapping solutions to distributions supported on the null cone C: Correspondingly to the various ways of generating solutions, we write down explicitly the unitary inner product and its Hilbert space.We have tried to avoid the use of any semi-simple theory and stay within classical analysis on spheres and Euclidean spaces; still our treatment may also be of interest to people working with the classification of the unitary dual of semi-simple Lie groups, since we are providing new models of some unipotent representations.Tools like the standard Knapp–Stein intertwining operators become very natural to use here, also from the more elementary viewpoint, and the close connection between these and Green functions for ultrahyperbolic differential operators seems not to have been noticed before.

2. Ultrahyperbolic equation onR^p1;q1 and conformal group

2.1. As explained in the Introduction, we shall give a flat picture, the so-called N-picture, of the minimal representation, which is connected to classical facts about

conformal geometry inRⁿ:We shall give a unitary inner product in this realization (see Theorem 6.2) and also in its Fourier transform (Theorem 4.9), together with an explicit form of minimalK-type in this realization (see Theorem 5.5).

We shall assume pþqA2N; pX2; qX2 and ðp;qÞað2;2Þ: The parity condition pþqA2N is not necessary when we consider a representation of the parabolic subgroup P^max or of the Lie algebra g: Indeed, it will be interesting to relax this parity condition in order to obtain an infinitesimally unitary representation, which does not integrate to a global unitary representation ofG:

Throughout this paper, we let

n¼pþq2:

This section is written in an elementary way, intended also for non-specialists of semisimple Lie groups. Sections 2.2 and 2.6 review the needed results in[11].

2.2. We recall some basic fact of the distinguished representation of a conformal group (see [11, Section 2]).Let M be an n-dimensional manifold with pseudo- Riemannian metricg_M:We denote byD_M the Laplace operator onM;and byK_M the scalar curvature ofM:The Yamabe operator is defined to be

D*_M :¼D_M n2 4ðn1ÞK_M:

Suppose ðM;gMÞ and ðN;gNÞ are pseudo-Riemannian manifolds.A local diffeomorphismF:M-Nis called aconformal mapif there exists a positive-valued functionOon M such that Fg_N ¼O²g_M: ForlAC; we introduce a twisted pull- back

F_l:C^NðNÞ-C^NðMÞ; f/O^lf3F: ð2:2:1Þ Then the conformal quasi-invariance of the Yamabe operator is expressed by

F_nþ2

2

D*N¼D*MF_n2

2

: ð2:2:2Þ

Let G be a Lie group acting conformally on M: If we write the action as x/L_hx ðhAG;xAMÞ;we have a positive functionOðh;xÞAC^NðGMÞsuch that

L_hg_M ¼Oðh;Þ²g_M ðhAGÞ:

We form a representation$_l ofG;with parameter lAC;onC^NðMÞas follows:

$lðh¹ÞfðxÞ ¼Oðh;xÞ^lfðLhxÞ; ðhAG;fAC^NðMÞ;xAMÞ: ð2:2:3Þ Note that the right-hand side is given by the twisted pull-back ðLhÞ_l according to notation (2.2.1). Then, Formula (2.2.2) implies that D*M:C^NðMÞ-C^NðMÞ is a

G-intertwining operator from $n2 2

to $nþ2 2

: Thus, we have constructed a distinguished representation of the conformal group:

Lemma 2.2 (see Kobayashi and Ørsted[11, Theorem 2.5]).KerD*M is a representa- tion space of the conformal group G of a pseudo-Riemannian manifold ðM;gMÞ;

through$n2 2

:

IfðN;g_NÞis also a pseudo-Riemannian manifold on which the same groupGacts conformally, then one can also define a representation$_l;N on C^NðNÞ: Then the twisted pull-backF_l is aG-intertwining operator.

2.3. Here is a setup on which we construct the minimal representation of Oðp;qÞby applying Lemma 2.2. Letp;qX2:We noten¼pþq2:We writefe0;y;enþ1gfor a standard basis ofR^pþq and the corresponding coordinate as

ðv0;y;v_nþ1Þ ¼ ðx;yÞ ¼ ðv0;z⁰;z⁰⁰;v_nþ1Þ;

where xAR^p;yAR^q;z⁰AR^p1;z⁰⁰AR^q1: The notation ðx;yÞwill be used for S^p1 S^q1;whileðz⁰;z⁰⁰ÞforRⁿ¼R^{ðp1Þþðq1Þ}:The standard norm onR^l will be written as j j ðl¼p1;p;q1;qÞ:

We denote byR^p;q the pseudo-Riemannian manifold R^pþq equipped with the flat pseudo-Riemannian metric:

g_R^p;q¼dv²₀þ?þdv²_p1dv²_p?dv²_nþ1: ð2:3:1Þ We put two functions onR^pþq by

n:R^pþq-R; ðx;yÞ/jxj; ð2:3:2Þ

m:R^pþq-R; ðv0;y;vnþ1Þ/¹₂ðv0þvnþ1Þ ð2:3:3Þ and define three submanifolds ofR^p;q by

X:¼ fðx;yÞAR^p;q:jxj ¼ jyja0g;

M:¼ fvAR^p;q:nðvÞ ¼1g-X¼S^p1S^q1; N:¼ fvAR^p;q:mðvÞ ¼1g-X’^B

i Rⁿ;

where the bijectioni:Rⁿ-N is given by

i:Rⁿ-N; ðz⁰;z⁰⁰Þ/ 1jz⁰j² jz⁰⁰j²

4 ;z⁰;z⁰⁰;1þjz⁰j² jz⁰⁰j² 4

!

: ð2:3:4Þ We say a hypersurfaceLofXistransversal to raysif the projection

F:X-M; v/ v

nðvÞ ð2:3:5Þ

induces a local diffeomorphism Fj_L:L-M: Then, one can define a pseudo- Riemannian metricgLof signatureðp1;q1ÞonLby the restriction of gR^p;q:In particular,M itself is transversal to rays, and the induced metric g_Sp1S^q1 equals g_Sp1"ðgS^q1Þ; where g_Sn1 denotes the standard Riemannian metric on the unit sphere Sⁿ¹: Likewise, the induced pseudo-Riemannian metric on Rⁿ through i:Rⁿ+R^p;q coincides with the standard flat pseudo-Riemannian metric g_R^p1;q1 onRⁿ:

2.4. Let Ip;q:¼diagð1;y;1;1;y;1ÞAGLðpþq;RÞ: The indefinite orthogonal group

G¼Oðp;qÞ:¼ fgAGLðpþq;RÞ:^tgI_p;qg¼I_p;qg;

acts isometrically on R^p;q by the natural representation, denoted by v/gv: This action stabilizes the light cone X: We note that the multiplicative group R_þ:¼ frAR:r40galso acts onXas dilation, which commutes with the linear action ofG:

Then, using dilation, one can define an action ofGon M;and also a meromorphic action onR^p1;q1 as follows:

L_h:M-M; v/ hv

nðhvÞ ðhAGÞ; ð2:4:1Þ

Lh:R^p1;q1-R^p1;q1; z/i¹ hiðzÞ mðhiðzÞÞ

ðhAGÞ: ð2:4:2Þ Then, both of these actions are conformal:

ðLhÞg_M ¼nðhvÞ²g_M; ð2:4:3Þ

ðLhÞg_R^p1;q1¼mðhiðzÞÞ²g_R^p1;q1: ð2:4:4Þ

We note that (2.4.2) and (2.4.4) are well-defined ifmðhiðzÞÞa0:In fact,Gacts only meromorphically onR^p1;q1: An illustrative example for this feature is the linear fractional transformation of SLð2;CÞon P¹C¼C,fNg;which is a meromorphic

action onC:This example essentially coincides with (2.4.2) for ðp;qÞ ¼ ð3;1Þ;since SLð2;CÞis locally isomorphic to Oð3;1ÞandCCR²:

2.5. The (meromorphic) conformal groups for the submanifoldsMandNofXare the same, namely,G¼Oðp;qÞ;while their isometry groups are different subgroups of G; as we shall see in Observation 2.5. In order to describe them, we define subgroupsK;M^max;N^max;A^max andN^max ofG as follows:

First, we set

m₀ :¼ Ipþq;

K :¼G-OðpþqÞ ¼OðpÞ OðqÞ;

M_þ^max :¼ fgAG:ge₀ ¼e₀; genþ1 ¼enþ1g COðp1;q1Þ;

M^max :¼M_þ^max,m0M_þ^max COðp1;q1Þ Z2: The Lie algebra ofG is denoted byg₀¼oðp;qÞ;which is given by matrices:

g₀CfXAMðpþq;RÞ:XI_p;qþI_p;q^tX¼Og:

Next, we keepn¼pþq2 in mind and put

E_j¼ 1 ð1pjpp1Þ;

1 ðppjpnÞ;

(

ð2:5:1Þ

and define elements ofg₀ as follows:

N%j:¼Ej;0þEj;nþ1EjE0;jþEjEnþ1;j ð1pjpnÞ;

ð2:5:2bÞNj:¼E_j;0E_j;nþ1EjE_0;jEjE_nþ1;j ð1pjpnÞ;

ð2:5:2cÞE:¼E0;nþ1þEnþ1;0; ð2:5:2aÞ whereEijdenotes the matrix unit.Now, we define abelian subgroups of Gby

N^max:¼exp Xⁿ

j¼1

RN%j

!

; N^max:¼exp Xⁿ

j¼1

RNj

!

; A^max:¼expðREÞ:

For example, M_þ^max is the Lorentz group and M_þ^maxN^max is the Poincare´ group if ðp;qÞ ¼ ð2;4Þ:It is convenient to identifyRⁿ withN^max by putting

%

na:¼exp Xⁿ

j¼1

ajNj

!

AN^max for a¼ ða1;y;anÞARⁿ: ð2:5:3Þ

The geometric point here will be the following:

Observation 2.5. (1) On S^p1S^q1;G acts conformally,while K isometrically.

(2) OnR^p1;q1; G acts meromorphically and conformally, while the motion group M_þ^maxN^max isometrically.

2.6. Next, let us consider the pseudo-Riemannian manifold M¼S^p1S^q1: It follows from (2.3.3) and (2.4.3) that we can define a representation$_l;M of G on C^NðMÞby

ð$l;Mðh¹ÞfÞðvÞ:¼nðhvÞ^lfðLhvÞ:

The Yamabe operator onM is of the form:

D*M ¼D_S^p1D_S^q1 p2 2

2

þ q2 2

2

¼D*_S^p1D*_S^q1:

Applying Lemma 2.2, we obtain a representation of the conformal group G¼ Oðp;qÞ;denoted byð$^p;q;V^p;qÞ;as a subrepresentation of$pþq4

2 ;M: V^p;q:¼KerD*M ¼ ffAC^NðMÞ:D*Mf ¼0g;

ð$^p;qðh¹ÞfÞðvÞ:¼nðhvÞ^pþq42 fðLhvÞ for hAG; vAM; fAV^p;q: The restriction of$^p;qfrom the conformal group to the isometry group gives useful knowledge on the representation $^p;q: For this, we recall the classical theory of spherical harmonics, which is a generalization of Fourier series forS¹:ForpX2 and kAN;we define the space of spherical harmonics of degreekby

H^kðR^pÞ ¼ ffAC^NðS^p1Þ:D_S^p1f ¼ kðkþp2Þfg;

¼ fAC^NðS^p1Þ:D*_S^p1f ¼ 1

4 kþp2 2

2!

f

( )

: ð2:6:1Þ Then OðpÞacts irreducibly onH^kðR^pÞand the algebraic direct sum"^N_k¼0H^kðR^pÞis dense inC^NðS^p1Þ:We note thatH^kðR²Þaf0gonly ifk¼0 or 1:

Now, we review a basic property of this representation ð$^p;q;V^p;qÞ on M¼S^p1S^q1:

Lemma 2.6 (See Binegar and Zierau[3]; Kobayashi and Ørsted [11, Section 3]).

Assume p;qX2;pþqA2Nand ðp;qÞað2;2Þ:

(1) ð$^p;q;V^p;qÞis an infinite dimensional irreducible representation of G:

(2) (K-type formula)V^p;q contains the algebraic direct sum

"

a;bAN aþp

2¼bþq 2

H^aðR^pÞ#H^bðR^qÞ ð2:6:2Þ

as a dense subspace with respect to the Fre´chet topology on C^NðMÞ:

(3) G preserves the norm on V^p;q defined by

jjFjj²_M :¼ 1 4D*_S^p1

¹₄

F

2

L²ðMÞ

¼ X

aXmaxð0;pq 2 Þ

aþq2 2

jjFa;bjj²_L2ðMÞ;

if F¼P

aF_a;bAV^p;q with F_a;bAH^aðR^pÞ#H^bðR^qÞ and b¼aþ^pq₂ : Here, ð¹₄ D*_S^p1Þ¹⁴ is a pseudo-differential operator on M;which is equal to ð¹₄D*_S^q1Þ¹⁴ on KerD*M:

We write ð ; Þ_M for the corresponding inner product.We denote by V^p;q the Hilbert completion ofV^p;q;on whichGacts as an irreducible unitary representation ofG:We shall use the same notation$^p;q to denote this unitary representation.

IfpXq then V^p;q contains theK-type of the form 12H^pq2 ðR^qÞ:This K-type is called a minimal K-type in the sense of Vogan, namely, its highest weight (with respect to a fixed positive root system ofk0) attains the minimum distance from the sum of negative roots ofk0 among all highest weights ofK-types occurring in$^p;q: Likewise forpoq:

Remark. (1) If pþqX8; $^p;q is called the minimal representation in the representation theory of semisimple Lie groups, in the sense that the annihilator is the Joseph ideal.

(2) Formula (2.6.2) is regarded as a branching law from the conformal groupGto the isometry subgroupKof the pseudo-Riemannian manifoldM¼S^p1S^q1(see Observation 2.5). In[12], we generalized this branching law with respect to a non- compact reductive subgroup and proved the Parseval–Plancherel formula, in the framework of discretely decomposable restrictions[10].

2.7. Let us consider the flat pseudo-Riemannian manifold R^p1;q1: The Yamabe operator onR^p1;q1 is of the form:

&_Rp1;q1&_z:¼ @²

@z²₁þ?þ @²

@z²_p1 @²

@z²_p? @²

@z²_pþq2;

because the scalar curvature onR^p1;q1vanishes.SinceG¼Oðp;qÞacts onR^p1;q1 as a (meromorphic) conformal transform by (2.4.4), we obtain a ‘representation’

with parameterlACas in (2.2.3):

$l;e;Rⁿðg¹ÞfðzÞ ¼ jmðgiðzÞÞj^lw_eðsgnðmðgiðzÞÞÞÞfðLgzÞ; ðgAGÞ: ð2:7:1Þ Here, fore¼71;we put

w_e:R-f71g

byw₁1 andw₁¼sgn:We may write$_l;e;R^p1;q1for$l;e;Rⁿ if we emphasize a view point of conformal geometry on the flat spaceR^p1;q1:

We note that C^NðMÞ is not stable by$_l;e;R^p1;q1ðg¹Þ because Lg is meromor- phic. To make (2.7.1) a representation, we need to consider suitable class of functions controlled at infinity.One method for this is to use a conformal compactification

R^p1;q1+ðS^p1S^q1Þ=BZ2;

and to take a twisted pull-back C_l from C^NðMÞ by a conformal map C:

This method is easy, and we shall explain it soon in Sections 2.8 and 2.9. The other is to find an inner product for specific parameter l so that G acts as a continuous unitary representation on the Hilbert space.This is particularly non- trivial for a subrepresentation, and we shall consider it for Ker&_Rp1;q1in Section 6.

Before taking a suitable class of functions, we first write a more explicit form of (2.7.1). First, we note that the maximal parabolic subgroup

P^max:¼A^maxM^maxN^max¼ ðR_þOðp1;q1Þ Z2ÞrRⁿ

acts transitively on the manifold iðRⁿÞ as affine transformations.Furthermore, M^maxN^max acts oniðRⁿÞas isometries (see Observation 2.5). Correspondingly, the representation$_l;e$_l;e;Rⁿgiven in (2.7.1) has a simple form when restricted to the subgroupP^max:

ð$l;eðmÞfÞðzÞ ¼fðm¹zÞ ðmAM_þ^maxÞ; ð2:7:2aÞ

ð$_l;eðm0ÞfÞðzÞ ¼efðzÞ;

ð$l;eðe^tEÞfÞðzÞ ¼e^ltfðe^tzÞ ðtARÞ;

ð2:7:2cÞð$l;eðn%aÞfÞðzÞ ¼fðz2aÞ ðaARⁿÞ: ð2:7:2bÞ Second, we write an explicit formula of the differential action of (2.7.1). We define a linear map

o:g₀-C^NðRⁿÞ

by the Lie derivative of the conformal factorOðh;zÞ:¼mðhiðzÞÞ¹(see (2.4.4)). For Y¼ ðYi;jÞ_0pi;jpnþ1Ag₀ andzARⁿ;we have

oðYÞ_z:¼d dt

_t¼0Oðe^tY;zÞ ¼ Y0;nþ11 2

Xⁿ

j¼1

ðY0;jþYnþ1;jÞzj: ð2:7:3Þ

We write the Euler vector field onRⁿas

Ez¼Xⁿ

j¼1

zj

@

@zj

: ð2:7:4Þ

Then the differentiald$l:g₀-EndðC^NðRⁿÞÞis given by d$lðYÞ ¼ loðYÞ oðYÞEz

Xⁿ

i¼1

ðYi;0þYi;nþ1Þ þjz⁰j² jz⁰⁰j²

4 ðYi;0þYi;nþ1Þ (

þXⁿ

j¼1

Yi;jzj

) @

@zj

ð2:7:5Þ

forY¼ ðYi;jÞ_0pi;jpnþ1Ag₀ andzARⁿ:In particular, we have d$lðNjÞ ¼ lEjz_jE_jz_jE_zþ1

2ðjz⁰j² jz⁰⁰j²Þ@

@zj

; ð1pjpnÞ: ð2:7:6Þ

2.8. We recall M¼S^p1S^q1: This subsection relates the representation $l;M

and$_l;R^p1;q1 by the stereographic projectionC¹:M-R^p1;q1 defined below.

We set a positive valued functiont:Rⁿ-Rby tðzÞ tðz⁰;z⁰⁰Þ:¼n3iðzÞ

¼ 1jz⁰j² jz⁰⁰j² 4

!2

þjz⁰j² 0

@

1 A

1 2

¼ 1þjz⁰j² jz⁰⁰j² 4

!2

þjz⁰⁰j² 0

@

1 A

1 2

¼ 1þ jz⁰j þ jz⁰⁰j 2

2!¹₂

1þ jz⁰j jz⁰⁰j 2

2!¹₂

: ð2:8:1Þ

We define an injective diffeomorphism as a composition of i:R^p1;q1+X (see (2.3.4)) andF:X-M (see (2.3.5)):

C:R^p1;q1-M; z/tðzÞ¹iðzÞ: ð2:8:2Þ

The image ofCis

Mþ:¼ fu¼ ðu0;u⁰;u⁰⁰;unþ1ÞAM¼S^p1S^q1:u₀þunþ140g: ð2:8:3Þ Then,Cis a conformal map (see[11, Lemma 3.3], for example) such that

CgM ¼tðzÞ²g_R^p1;q1: ð2:8:4Þ

The inverse ofC:R^p1;q1-Mþis given by C¹ðu0;u⁰;u⁰⁰;unþ1Þ ¼ u₀þunþ1

2

¹

ðu⁰;u⁰⁰Þ ¼mðuÞ¹ðu⁰;u⁰⁰Þ: ð2:8:5Þ C¹ is nothing but a stereographic projection if q¼1: We note thatC induces a conformal compactification of the flat spaceR^p1;q1:

R^p1;q1+ðS^p1S^q1Þ=BZ₂:

HereBZ2 denotes the equivalence relation in the direct product spaceS^p1S^q1 defined byuBu:

As in (2.2.1), we define the twisted pull-back by

C_l:C^NðMÞ-C^NðRⁿÞ; F/tðzÞ^lFðCðzÞÞ: ð2:8:6Þ Let

C^NðMÞ_e:¼ ffAC^NðMÞ:fðuÞ ¼efðuÞ; for any uAMg:

ThenC_lj_C^N_ðMÞ

e is injective.The inverse map is given by ðC_l;eÞ¹:C_lðC^NðMÞ_eÞ-C^NðMÞ_e;

f/

u₀þu_nþ1 2

l

fðC¹ðuÞÞ ðuAMþÞ;

eu0þunþ1

2

l

fðC¹ðuÞÞ ðuAMÞ: 8>

><

>>

:

ð2:8:7Þ

We note thatðC_l;eÞ¹f makes sense for fAC^N₀ ðRⁿÞ;since we have C₀^NðRⁿÞCC_lðC^NðMÞ_eÞ:

Now, the representation$l;e;Rⁿis well-defined on the following representation space:

C_lðC^NðMÞÞ;a subspace ofC^NðRⁿÞthrough$l;M: Then, by (2.2.2) (see[11, Proposition 2.6]), we have:

Lemma 2.8. C_n2

2

ðV^p;qÞCKer&_Rp1;q1; where V^p;q¼KerD*M:

2.9. In the terminology of representation theory of semisimple Lie groups, C_l is a G-intertwining operator from the K-picture ð$_l;M;C^NðMÞ_eÞ to the N-picture ð$l;e;Rⁿ;C_lðC^NðMÞÞÞ: To see this in an elementary way, we argue as follows: For nAC;we denote by the space

SⁿðXÞ:¼ fhAC^NðXÞ:hðtxÞ ¼tⁿhðxÞ for any xAX;t40g; ð2:9:1Þ

of smooth functions onXof homogeneous degree n: ThenG acts onSⁿðXÞby left translations.Furthermore, fore¼71;we put

S^n;eðXÞ:¼ fhASⁿðXÞ:hðxÞ ¼ehðxÞ; for any xAXg: ð2:9:2Þ Then we have a direct sum decomposition

SⁿðXÞ ¼S^n;1ðXÞ þS^n;1ðXÞ;

on whichG acts by left translations, respectively.Then S^n;eðXÞcorresponds to the degenerate principal series representation (see[11]for notation):

C^N-Ind^G_Pmaxðe#ClÞCS^ln2;eðXÞ; ð2:9:3Þ whereP^max¼M^maxA^maxN^max:

Lemma 2.9. (1)The restriction S^l;eðXÞ-C^NðMÞ_e;h/hj_M induces the isomorphism of G-modules between S^l;eðXÞand ð$l;C^NðMÞ_eÞfor anylAC:

(2) The restriction S^l;eðXÞ-C^NðRⁿÞ;h/hj_Rⁿ induces the isomorphism of G- modules between S^l;eðXÞandð$l;e;Rⁿ;C_lðC^NðMÞ_eÞÞfor anylAC:

Proof. See [11, Lemma 3.7.1]for (1).Statement (2) follows from the commutative diagram:

ð2:9:4Þ

becauser₁ is bijective andr₂ is injective. &

2.10. A natural bilinear form/ ; S:S^ln2ðXÞ S^ln2ðXÞ-Cis defined by /h1;h2S:¼

Z

M

h1ðbÞh2ðbÞdb ð2:10:1Þ

¼2 Z

Rⁿ

h₁ðiðzÞÞh2ðiðzÞÞdz ðsee ð2:3:4ÞÞ: ð2:10:2Þ

Here, db is the Riemannian measure on M¼S^p1S^q1: The second equation follows from ðh1h2ÞðiðzÞÞ ¼tðzÞⁿðh1h2ÞðCðzÞÞand the Jacobian for C:Rⁿ-M is given bytðzÞⁿ (see (2.8.4)). Then /; S is K-invariant and N^max-invariant from (2.10.1) and (2.10.2), and thusG-invariant sinceGis generated by K andN^max:

3. Square integrable functions on the cone

3.1. In this section, we shall study the irreducible unitary representation of the motion group M_þ^maxN^maxCOðp1;q1ÞrR^pþq2 and the maximal parabolic subgroup P^max¼M^maxA^maxN^max on the space of solutions to our ultrahyperbolic equation

&_Rp1;q1f ¼0:This is a standard induced representation by the Mackey machine, and

will be later extended to the minimal representation ofG¼Oðp;qÞ(see Theorem 4.9(3)).

3.2. In the flat picture R^p1:q1; our minimal representation V^p;q of Oðp;qÞcan be realized in a subspace of Ker&_Rp1;q1 (see Lemma 2.8). We shall study the representation space by means of the Fourier transform.

We normalize the Fourier transform onSðRⁿÞby ðFfÞðzÞ ¼

Z

Rⁿ

fðzÞe^pffiffiffiffiffi_1ðz

1z1þ?þznznÞ

dz1?dzn; and extends it toS⁰ðRⁿÞ;the space of the Schwartz distributions.

By composing the following two injective maps:

C^NðMÞ_e !

Cn2

2 C^NðRⁿÞ-S⁰ðRⁿÞ !^FS⁰ðRⁿÞ;

we define a representation ofG and g on the image FC_n2

2

ðC^NðMÞÞ;denoted by

#

$l;e$#l;e;Rⁿ; so that F3C_n2

2

is a bijective G-intertwining operator from the representation spaceð$l;M;C^NðMÞ_eÞtoð$#l;e;FC_n2

2

ðC^NðMÞ_eÞÞ:Then, it follows from (2.7.2) that the representation $#_l;e has a simple form when restricted to the subgroupP^max¼M^maxA^maxN^max:

ð$#_l;eðmÞhÞðzÞ ¼hð^tmzÞ ðmAM_þ^maxÞ; ð3:2:1aÞ ð$#l;eðm0ÞhÞðzÞ ¼ehðzÞ;

ð$#_l;eðe^tEÞhÞðzÞ ¼e^ðlnÞthðe^tzÞ ðtARÞ; ð3:2:1bÞ ð$#l;eðn%aÞhÞðzÞ ¼e^2pffiffiffiffiffi_1ða

1z1þ?þanznÞhðzÞ ðaARⁿÞ: ð3:2:1cÞ

We remark that in the above formula, we regardedhas a function.The action of A^max on the space of distributions is slightly different by the contribution of the measuredz:

ð$#_l;eðe^tEÞfÞðzÞ ¼e^ltfðe^tzÞ ðtARÞ; ð3:2:1b⁰Þ if we writefðzÞ ¼hðzÞdzAS⁰ðRⁿÞ:

The differential representation d$#_l;e of g₀ on FC_n2

2

ðC^NðMÞÞ is given by the following lemma:

Lemma 3.2. We recall that E_z is the Euler operator (see (2.7.4)). With notation in (2.5.2),we have

d$#_l;eðN%_jÞ ¼2 ffiffiffiffiffiffiffi p1

z_j ð1pjpnÞ;

d$#l;eðNjÞ ¼ ffiffiffiffiffiffiffi p1

ðlnÞEj

@

@z_jEzEj

@

@z_jþ1 2z_j&_z

ð1pjpnÞ; d$#_l;eðEÞ ¼lnE_z:

Proof. Lemma follows from the correspondence under the Fourier transform

@

@zj2 ffiffiffiffiffiffiffi p1

z_j; zj2 ffiffiffiffiffiffiffi p1_@

@zj; and therefore from Ez2nEz; PðzÞ2&_z; wherePðzÞ:¼z²₁þ?þz²_p1z²_p?z²_pþq2: &

Remark. (1) We note thatd$#l;e is independent of the signaturee¼71:

(2) In Theorem 4.9, we shall find thatL²ðCÞ;the Hilbert space of square integrable functions on the cone C in Rⁿ; is a G-invariant subspace of the Schwartz distributionsS⁰ðRⁿÞ:Then, the action of the Lie algebragcan be written in terms of differential operators along the coneCat most of second order.

3.3. We define a quadratic formQonRⁿ ðCðRⁿÞÞas the dual ofPðzÞon Rⁿ by QðzÞ:¼z²₁þ?þz²_p1z²_p?z²_pþq2

for zARⁿ¼R^pþq2 ð3:3:1Þ

and define a closed cone by

C:¼ fzARⁿ:QðzÞ ¼0g: ð3:3:2Þ It follows from Lemma 2.8 that the support of the distributionFC_n2

2

Fis contained in the coneC;for anyFAV^p;q:Surprisingly,FC_n2

2

F becomes square integrable on C(see Theorem 4.9). As a preparation for the proof, we study a natural action on L²ðCÞof a parabolic subgroupP^max in this subsection.

We take a differentialðn1Þ-form dm onC such that dQ4dm¼dz₁4?4dz_n:

Then the restrictiondmto the coneC defines a canonical measure (we use the same notation dm).Using polar coordinates on C: z¼ ðso;so⁰Þ with s40; oAS^p2; o⁰AS^q2;we write down the canonical measuredmonC explicitly by

Z

C

fdm¼1 2

Z N 0

Z

S^p2

Z

S^q2

fðso;so⁰Þsⁿ³ds dodo⁰ ð3:3:3Þ for a test function f: If n42; that is if pþq44; then the measure dm defines a Schwartz distribution onRⁿ;for which we shall also writedðQÞ;the ‘‘delta function’’

supported on the coneC(see[7]).

For a measurable functionfon C;we define a norm of fby jjfjj² :¼

Z

C

jfj²dm ð3:3:4Þ

and denote byL²ðCÞ L²ðC;dmÞthe Hilbert space of square integrable functions.

By the Mackey theory, we can define a natural representationp of the maximal parabolic subgroupP^max¼A^maxM^maxN^max onL²ðCÞifpþqA2Z;by

ðpðe^tEÞcÞðzÞ:¼e^{n22 t}cðe^tzÞ ðtARÞ;

ð3:3:5bÞðpðmÞcÞðzÞ:¼cð^tmzÞ ðmAM_þ^maxÞ; ð3:3:5aÞ

ðpðm0ÞcÞðzÞ:¼ ð1Þ

pq 2 cðzÞ;

ðpðn%aÞcÞðzÞ:¼e^2pffiffiffiffiffi_1ða

1z1þ?þanznÞcðzÞ; ðaARⁿÞ: ð3:3:5cÞ

Proposition 3.3. (1) The representationðp;L²ðCÞÞof P^max is unitary.

(2) The representation p is still irreducible when restricted to the motion group M_þ^maxN^maxCOðp1;q1ÞrR^pþq2: In particular, it is irreducible as a P^max- module.

Proof. Statement (1) is straightforward from definitions (3.3.4) and (3.3.5).

Let us prove (2). It follows from (3.3.5)(c) that anyN^max-invariant closed subspace ofL²ðCÞis of the formL²ðC⁰ÞwhereC⁰is a measurable subset ofC:AsM_þ^max acts transitively onC;L²ðC⁰ÞisM_þ^max-invariant only if the measure ofC⁰is either null or

conull.Thus,L²ðC⁰Þequals eitherf0g orL²ðCÞ:Therefore, the unitary representa- tionL²ðCÞis irreducible as anM_þ^maxN^max-module. &

3.4. It is not clear a priori ifðp;L²ðCÞÞextends fromP^max toG:We shall prove in Theorem 4.9 that if p;qX2 and nð¼pþq2Þ42 then p extends to G as an irreducible unitary representation through an injective map T:L²ðCÞ+S⁰ðRⁿÞ;

defined as follows:

By using the Cauchy–Schwarz inequality, we see the following map TðcÞ:SðRⁿÞ-C; j/Z

C

jc dm

is well-defined and continuous ifn42;for each cAL²ðCÞ: Thus we have a natural map

T:L²ðCÞ-S⁰ðRⁿÞ; c/cdm: ð3:4:1Þ Clearly,T is injective.We shall regardTðL²ðCÞÞas a Hilbert space such thatT is a unitary operator.

Lemma 3.4. (1) T is a P^max-intertwining operator from ðp;L²ðCÞÞ to ð$#n2

2 ;ej_Pmax;S⁰ðRⁿÞÞ:

(2)ð$#n2

2 ;ej_Pmax;TðL²ðCÞÞÞis an irreducible unitary representation of P^max:It is still irreducible as an M^maxN^max-module.

Proof. Statement (1) follows directly from definitions (3.2.1) and (3.3.5). Statement (2) follows from (1) and Proposition 3.3 (2). &

4. Green function and inner product

4.1. In this section, we shall construct solutions of the ultrahyperbolic equation

&_Rp1;q1f ¼0 by the integral transform given by convolution with the Green kernel.

Then, we shall show that the Green kernel coincides with a special value of the Knapp–Stein intertwining operator for a degenerate principal series.This observa- tions gives another expression of the inner product of the minimal representation of Oðp;qÞby using the Green kernel (see Theorem 4.7), and also leads to a realization of the minimal representation on L²ðCÞ; the Hilbert space of square integrable functions on a coneCas will be discussed in Section 6.

We put

PðxÞ ¼x²₁þ?þx²_p1x²_p?x²_n forxARⁿ¼R^pþq2: