Alice Devillers, Cai Heng Li, Geoff Pearce, Cheryl Praeger and I have just uploaded to the arxiv a preprint of our recently submitted paper On imprimitive rank 3 permutation groups‘.

A permutation group ${G}$ on a set ${\Omega}$ also acts on the set ${\Omega\times\Omega}$ via ${(\omega_1,\omega_2)^g=(\omega_1^g,\omega_2^g)}$. If ${G}$ is transitive on ${\Omega}$ then we cannot expect it to be transitive on ${\Omega\times\Omega}$ as the subset ${\{(\omega,\omega)\mid\omega\in\Omega\}}$ is an orbit. The best that we can hope for is that ${G}$ has two orbits on ${\Omega\times\Omega}$ and this is equivalent to ${G}$ acting 2-transitively on ${\Omega}$, that is, ${G}$ acts transitively on the set of ordered pairs of distinct elements of ${G}$. The rank of a permutation group ${G}$ is the number of orbits of ${G}$ on ${\Omega\times\Omega}$. Thus 2-transitive groups have rank 2.

The number of orbits of ${G}$ on ${\Omega\times \Omega}$ is equal to the number of orbits of the point stabiliser ${G_{\omega}}$ on ${\Omega}$. Indeed, given an orbit ${\Delta}$ of ${G_{\omega}}$ on ${\Omega}$ the set ${\{(\omega,\delta)^g\mid \delta\in\Delta,g\in G\}}$ is an orbit of ${G}$ on ${\Omega\times\Omega}$. Conversely, given an orbit ${\Delta}$ of ${G}$ on ${\Omega\times\Omega}$, the set ${\{\delta\in\Omega\mid (\omega,\delta)\in\Delta\}}$ is an orbit of ${G_{\omega}}$ on ${\Omega}$. In this set up, ${\{\omega\}}$ corresponds to ${\{(\omega,\omega)\mid\omega\in\Omega\}}$.

The study of rank 3 permutation groups goes back to Donald G. Higman in his paper Finite permutation groups of rank 3′, Math. Z. 86, 145–156 (1964). A rank 3 group ${G}$ may be primitive or imprimitive. If it is primitive there are three possibilities:

1. ${G}$ is an almost simple group,
2. ${G}$ is a subgroup of the affine group ${\mathrm{AGL}(d,p)}$ for some prime ${p}$
3. ${G\leqslant H\mathrm{wr} S_2}$ acting on ${\Delta^2}$ in product action where ${H}$ is a 2-transitive group on the set ${\Delta}$.

After the Classification of 2-transitive groups, work by Bannai in the alternating group case, Kantor and Liebler in the classical group case, Liebeck in the affine group case, and Liebeck and Saxl in the remaining cases of the sporadics and exceptional groups, all primitive rank 3 groups are known.

If ${G}$ is imprimitive on ${\Omega}$ with a system of imprimitivity ${\mathcal{B}}$ with ${n}$ parts, then ${G}$ must be 2-transitive on ${\mathcal{B}}$ and for ${B\in\mathcal{B}}$, ${G_B}$ is 2-transitive on ${B}$. Moreover, Higman showed that ${\mathcal{B}}$ is the unique system of imprimitivity for ${G}$. If we let ${H}$ be the permutation group induced by ${G_B}$ on ${B}$ and ${K}$ is the group induced by ${G}$ on ${\mathcal{B}}$, then using the “Embedding Theorem” for imprimitive groups we have that ${G}$ is a subgroup of ${H\mathrm{wr} K}$ in its imprimitive action.

All 2-transitive groups are known and by a classical theorem of Burnside are either almost simple or affine, that is, the unique minimal normal subgroup ${T}$ of ${H}$ is either nonabelian simple or elementary abelian. We can think of the rank 3 group ${G}$ to be “large” if it contains ${T^n}$ while it is “small” if ${G\cap H^n=1}$, that is, if ${G}$ acts faithfully on ${\mathcal{B}}$. The set of orbits of a normal subgroup of ${G}$ forms a system of imprimitivity. Since ${\mathcal{B}}$ is the unique system of imprimitivity for ${G}$ the latter case implies that all normal subgroups of ${G}$ are transitive on ${\Omega}$, that is, ${G}$ is quasiprimitive on ${\Omega}$.

Our first main theorem is that if ${G}$ is an imprimitive group with the above set up such that ${H}$ is an almost simple 2-transitive group then ${G}$ is rank 3 if and only if one of the following holds.

1. ${T^n\leqslant G}$, that is, ${G}$ is “large”,
2. ${G}$ is quasiprimitive and rank 3 on ${\Omega}$,
3. ${G=M_{10},\mathrm{PGL}(2,9),\mathrm{Aut}(A_6)}$ acting on 12 points,
4. ${G=\mathrm{Aut}(M_{12})}$ acting on 24 points.

We then classify all the quasiprimitive groups of rank 3 which are imprimitive. We show that they must all be almost simple groups and we find 8 individual groups plus certain actions of groups containing ${\mathrm{PSL}(d,q)}$ acting with ${\frac{q^d-1}{q-1}}$ blocks. Combined with the previous classification of all primitive rank 3 groups this implies that all quasiprimitive rank 3 groups are now known. The open case in the classification of all rank 3 groups is then the imprimitive case where the group induced on a block is a 2-transitive affine group. We say a little bit about this case in the paper.

We achieved our classification of quasiprimitive rank 3 groups by determining all almost simple groups ${G}$ which have a system of imprimitivity ${\mathcal{B}}$ such that ${G}$ acts 2-transitively on ${\mathcal{B}}$ and for ${B\in\mathcal{B}}$ we have ${G_B}$ is 2-transitive on ${B}$. As mentioned earlier this includes all rank 3 imprimitive groups which are almost simple.

We became interested in a classification of these groups during our work on locally 2-distance transitive graphs. These are graphs where the stabiliser of a vertex ${v}$ acts transitively on the set of vertices at distance 1 from ${v}$ and on the set of vertices at distance 2 from ${v}$. If the graph is also vertex-transitive then we refer to it as being 2-distance transitive. During our analysis, one case which arose was that of complete multipartite graphs with a group of automorphisms ${G}$ which acts faithfully on the partition into partite blocks. The group ${G}$ will be then be 2-distance transitive if it has rank 3. Our classification of quasiprimitive rank 3 groups which are imprimitive then gives a complete list of all possibilities which arise.

## 2 thoughts on “Rank 3 permutation groups”

1. Dear Michael

I hope you are doing well.

My question is as follows:

Is there a transitive group having three pairwise
inequivalent representations of rank 3 with the same permutation character?

I should mention that D.G. Higman implicitly indicated that he does not know whether or not such a group exists.

Best.
Reza

1. Michael Giudici says:

Hi Reza,

No we didn’t find any such groups. We did find that it was possible for a certain overgroup of PSL(2,q) to have two different rank 3 actions of degree 2(q+1) (see Proposition 5.13 of the paper).