I have decided that any mathematical blog entitled SymOmega should have an expository post about one of the most influential theorems of permutation group theory, that is, The O’Nan-Scott Theorem. Originally, this was a theorem about maximal subgroups of the symmetric group.  It appeared as an appendix to a paper of Leonard Scott in the proceedings of the Santa Cruz symposium on finite simple groups in 1979, with a footnote that Michael O’Nan had independently proved the same result. Apparently there are even earlier versions due to various people but it was the Classification of Finite Simple Groups (CFSG) which meant that it would become very useful.

In its simplest form the theorem states that a maximal subgroup of the symmetric group $\mathrm{Sym}(\Omega)$ where $|\Omega|=n$, is one of the following:

1. $S_k\times S_{n-k}$, the stabiliser of a k-set (that is, intransitive),
2. $S_a \mathrm{wr} S_b$ with $n=ab$, the stabiliser of a partition into b parts of size a (that is, imprimitive), or
3. primitive (that is, preserves no nontrivial partition) and of one of the following types:
• $\mathrm{AGL}(d,p)$,
• $S_\ell \mathrm{wr} S_k$ with $n=\ell^k$, the stabiliser of a product structure $\Omega=\Delta^k$,
• a group of diagonal type, or
• an almost simple group.

(These  types will be explained in further detail below).

It was soon recognised (perhaps first in a paper of Peter Cameron)  that the real power is in the ability to split the finite primitive groups into various types.  Problems concerning primitive groups can then be studied by solving them for each type.  This often sees questions about primitive groups reduced to questions about simple groups and then the force of the classification can be harnessed to get your result.  Furthermore, many results about transitive permutation groups can be reduced to the primitive case and so we actually have a tool for studying questions about transitive groups.

The division of primitive groups into various types is usually finer than given in the statement about maximal subgroups of $S_n$ as we are no longer concerned about maximality in $S_n$ but instead are more concerned about the types of actions. In Cameron’s original paper, the twisted wreath product case (again, more details later) which does not appear as a maximal subgroup of $S_n$ but is an important type of action with a distinctly different flavour to the maximal subgroup $S_\ell \mathrm{wr} S_k$  it is contained in,  was left out and this was corrected in a subsequent paper of Aschbacher and Scott, and in work of Laci Kovacs. A complete self-contained proof is given in a paper by Martin Liebeck, Cheryl Praeger and Jan Saxl.

Since I am from Perth, I usually  follow the  division and labelling into 8 types due to Cheryl Praeger which appears here and here.  First we need to deal with a few preliminaries.

Given a transitive permutation group G with nontrivial normal subgroup N, the orbits of N form a system of imprimitivity for G.  This can be easily seen as follows: let $\alpha\in\Omega$ and $g\in G$. Then $(\alpha^N)^g=\alpha^{Ng}=\alpha^{gN}=(\alpha^g)^N$ and so G maps N-orbits to N-orbits. Hence all nontrivial normal subgroups of a primitive group are transitive. A primitive group can have at most two minimal normal subgroups and if there are two then they are isomorphic and each is regular. In fact, any minimal normal subgroup is isomorphic to $T^k$ for some simple group T. The product of all minimal normal subgroups of a group is called its socle. The primitive groups can then be sorted by the structure and action of their minimal normal subgroups.

The eight O’Nan-Scott types are as follows:

HA: These are the primitive groups which are subgroups of the affine general linear group $\mathrm{AGL}(d,p)$, for some prime p and positive integer $d\geq 1$. For such a group  G to be primitive, it must contain the subgroup of all translations, and the stabiliser $G_0$ in G of the zero vector must be an irreducible subgroup of $\mathrm{GL}(d,p)$. Primitive groups of type HA are characterised by having a unique minimal normal subgroup which is elementary abelian and acts regularly. The HA stands for “holomorph of an abelian group”.

HS: Let T be a finite nonabelian simple group. Then $M= T\times T$ acts on $\Omega=T$ by $t^{(t_1,t_2)}=t_1^{-1}tt_2$. Now M has two minimal normal subgroups $N_1,N_2$, each isomorphic to T and each acts regularly on $\Omega$, one by right multiplication and one by left multiplication. The action of M is primitive and if we take $\alpha=1_T$ we have that $M_{\alpha}=\{(t,t)\mid t\in T\}$, which induces $\mathrm{Inn}(T)$ on $\Omega$. In fact any automorphism of T will act on $\Omega$.  A primitive group of type HS is then any group G such that $M\cong T.\mathrm{Inn}(T)\leqslant G\leqslant T.\mathrm{Aut}(T)$. All such groups have $N_1$ and $N_2$ as minimal normal subgroups.  The HS stands for “holomorph of a simple group”.

HC: Let T be a nonabelian simple group and let $N_1\cong N_2\cong T^k$ for some integer $k\geq 2$. Let $\Omega=T^k$. Then $M=N_1\times N_2$ acts transitively on $\Omega$ via $x^{(n_1,n_2)}=n_1^{-1}xn_2$ for all $x\in \Omega$, $n_1\in N_1$ and $n_2\in N_2$. As in the HS case, we have $M\cong T^k.\mathrm{Inn}(T^k)$ and any automorphism of $T^k$ also acts on $\Omega$. A primitive group of type HC is a group G such that $M\leqslant G\leqslant T^k.\mathrm{Aut}(T^k)$ and G induces a subgroup of $\mathrm{Aut}(T^k)=\mathrm{Aut}(T) \mathrm{wr} S_k$ which acts transitively on the set of k simple direct factors of $T^k$. Any such G has two minimal normal subgroups, each isomorphic to $T^k$ and regular. The HC stands for “holomorph of a compound group”.

A group of type HC preserves a product structure $\Omega=\Delta^k$ where $\Delta=T$ and $G\leqslant H\mathrm{wr} S_k$ where $H$ is a primitive group of type HS on$\Delta$.

TW: Here G has a unique minimal normal subgroup N and $N\cong T^k$ for some finite nonabelian simple group T and N acts regularly on $\Omega$. Such groups can be constructed as twisted wreath products and hence the label TW. The conditions required to get primitivity imply that $k\geq 6$ so the smallest degree of such a primitive group is $60^6$.

AS: Here G is a group lying between T and $\mathrm{Aut}(T)$, that is, G is an almost simple group and so the name. We are not told anything about what the action is, other than that it is primitive. Analysis of this type requires knowing about the possible primitive actions of almost simple groups, which is equivalent to knowing the maximal subgroups of almost simple groups.

SD: Let $N=T^k$ for some nonabelian simple group T and integer $k\geq 2$ and let $H=\{(t,\ldots,t)\mid t\in T\}\leqslant N$. Then N acts on the set $\Omega$ of right cosets of H in N by right multiplication. We can take $\{(t_1,\ldots,t_{k-1},1)\mid t_i\in T\}$ to be a set of coset representatives for H in N and so we can identify $\Omega$ with $T^{k-1}$.  Now $(s_1,\ldots,s_k)\in N$ takes the coset with representative $(t_1,\ldots,t_{k-1},1)$ to the coset $H(t_1s_1,\ldots,t_{k-1}s_{k-1},s_k)=H(s_k^{-1}t_1s_1,\ldots,s_k^{-1}t_{k-1}s_{k-1},1)$.  The group $S_k$ induces automorphisms of N by permuting the entries and fixes the subgroup H and so acts on the set $\Omega$. Also, note that H acts on $\Omega$ by inducing $\mathrm{Inn}(T)$ and in fact any automorphism $\sigma$ of T acts on $\Omega$ by taking the coset with representative $(t_1,\ldots,t_{k-1},1)$ to the coset with representative $(t_1^\sigma,\ldots,t_{k-1}^\sigma,1)$. Thus we get a group $W=N. (\mathrm{Out}(T)\times S_k)\leqslant \mathrm{Sym}(\Omega)$.  A primitive group of type SD is a group $G\leqslant W$ such that $N\lhd G$ and G induces a primitive subgroup of $S_k$ on the k simple direct factors of N. The label SD stands for “simple diagonal”.

CD: Here $\Omega=\Delta^k$ and $G\leq H\mathrm{wr} S_k$ where H is a primitive group of type SD on$\Delta$ with minimal normal subgroup $T^{\ell}$. Moreover, $N=T^{\ell k}$ is a minimal normal subgroup of G and G induces a transitive subgroup of $S_k$. The label CD stands for “compound diagonal”.

PA: Here $\Omega=\Delta^k$ and $G\leqslant H\mathrm{wr} S_k$ where H is a primitive almost simple group on$\Delta$ with socle T.  Thus G has a product action on $\Omega$ and hence the label PA. Moreover, $N=T^k\lhd G$ and G induces a transitive subgroup of $S_k$ in its action on the k simple direct factors of N.

As mentioned before some authors use different divisions of the types. The most common is to include types HS and SD together as a “diagonal type” and types HC, CD and PA together as a “product action type”.  Cameron uses the notion of a basic primitive group as one which does not preserve a product structure. Then the basic types are HA, HS, SD and AS. Some groups of our HA type can by nonbasic, for example subgroups of $\mathrm{AGL}(n,p)\mathrm{wr} S_k$ acting on $p^{nk}$ points.

As we have seen, all nontrivial normal subgroups of a primitive group are transitive. A permutation group is called quasiprimitive if all nontrivial normal subgroups are transitive. This is  weaker than primitive, as for example an action of a simple group is quasiprimitive, but not all are primitive. Praeger generalised the O’Nan-Scott Theorem to quasiprimitive groups. All such groups can be grouped into 8 types with the same labels as in the primitive case. Types HA, HS, HC stay as is and so only include primitive groups. The remaining types need slight tweaking as follows:

TW: To be quasiprimitive of this type is less restrictive and in fact we no longer have a restriction on the number of simple direct factors of N. In this type we include the action of T on itself by right multiplication.

SD: We only require that G induces a transitive subgroup of $S_k$ on the k simple direct factors of N, instead of a primitive one.

CD: We only require that H is a quasiprimitive group of type SD.

AS: We only require that the simple group T is transitive and not regular.

PA: This type has perhaps the greatest difference from its corresponding primitive type.  Here G acts on a set $\Omega$ and acts faithfully on a nontrivial partition $\mathcal{P}$ of $\Omega$ such that $\mathcal{P}$ can be identified with $\Delta^k$ for some $k\geq 2$, such that $G\cong G^{\mathcal{P}}\leqslant H\mathrm{wr} S_k$ where H is an almost simple group acting transitively on $\Delta$ with nonregular  socle T. Moreover, $T^k\lhd G$ and G induces a transitive subgroup of $S_k$. Thus G has a unique minimal normal subgroup $N\cong T^k$ and there exists $1\neq R\leqslant T$ and $B\in\mathcal{P}$ such that $N_B=R^k$. Then for $\alpha\in B$ the stabiliser $N_{\alpha}$ is a subgroup of $N_B$ which projects onto R in each coordinate.

I should end with one further generalisation of primitivity.  A permutation group G is called innately transitive if it has a transitive minimal normal subgroup. All quasiprimitive groups are innately transitive.  A simple example of an innately transitive group which is not quasiprimitive is $G= A_5\times R$, where $R\cong C_5\leqslant A_5$  acting on $\Omega=A_5$ such that $t^{(s,h)}=s^{-1}th$ for all $t,s\in A_5$ and $h\in C_5$.  Then R is an intransitive normal subgroup of G but G contains a transitive minimal normal subgroup isomorphic to $A_5$. There is nothing special about $A_5$ or R here so similar examples can be constructed from any nonabelian simple group. In his thesis, John analysed the actions of innately transitive groups and despite the fact that this seems to be a much weaker condition than primitivity was able to derive an O’Nan-Scott-like Theorem for them.

[Added 14/10/09:]  It has been pointed out to me that linking to mathscinet reviews may not be helpful if you don’t have a subscription so the details of the cited papers which I am unable to find free online copies of are as follows:

• M. Aschbacher and L. Scott, Maximal subgroups of finite groups, J. Algebra 92 (1985) 44–80.
• L.G. Kovács, Maximal subgroups in composite finite groups, J. Algebra 99 (1986) 114–131.
• Martin W. Liebeck, Cheryl E.  Praeger and Jan  Saxl, On the O’Nan-Scott theorem for finite primitive permutation groups, J. Austral. Math. Soc. Ser. A 44 (1988), 389–396.
• Cheryl E. Praeger, Finite quasiprimitive graphs in: Surveys in combinatorics, 1997 (London), 65–85, London Math. Soc. Lecture Note Ser., 241, Cambridge Univ. Press, Cambridge, 1997.
• Cheryl E. Praeger, Cai Heng Li and Alice C. Niemeyer, Finite transitive permutation groups and finite vertex-transitive graphs in:Graph symmetry (Montreal, PQ, 1996), 277–318, NATO Adv. Sci. Inst. Ser. C Math. Phys. Sci., 497, Kluwer Acad. Publ., Dordrecht, 1997.

## One thought on “The O’Nan-Scott Theorem”

ASQ (Almost Simple Quotient type): $N$ is a nonabelian simple group and $G = NG_\alpha$ where $G_\alpha$ is a point stabiliser and it doesn’t contain $N$. Moreover, the centraliser of $N$ in $G$ is nontrivial and $G\ne C_G(N)G_\alpha$.
PQ (Product Quotient type): $N=T^k$ is nonabelian, nonsimple, and $\Omega=N$. Now $G$ is contained in $N :\mathrm{Aut}(N)$ and $G$ is conjugate in $\mathrm{Sym}(N)$ to a subgroup of $V\, \mathrm{wr}\, S_k$ in product action, where $V$ is innately transitive of ASQ type with regular minimal normal subgroup $T$. The centraliser of $N$ in $G$ is nontrivial and not a subdirect subgroup of the left regular representation of $N$.
DQ (Diagonal Quotient type): Again, $N$ is nonabelian, nonsimple, $\Omega = N$ and $G$ is contained in $N : \mathrm{Aut}(N)$. The group $G$ is conjugate in $\mathrm{Sym}(N)$ to a subgroup of $V\, wr\, S_k$ in product action, where $V$ is primitive of HS with regular minimal normal subgroup $T$, and the centraliser of $N$ in $G$ is a proper subdirect subgroup of the left regular representation of $N$. Moreover $C_G(N)$ is a direct product of m full diagonal subgroups of $T^{k/m}$ for some proper divisor m of k, and $G \leq K:[(A\times S_{k/m})\,\mathrm{wr}\, S_m]$, where $A$ is a full diagonal subgroup of $latex\mathrm{Aut}(T)^{k/m}$, and the projection of $G_\alpha$ onto $S_{k/m}\,\mathrm{wr}\, S_m$ is transitive.
PA (Product Action type): The difference here is that $G$ need not act faithfully on $\mathcal{P}$ and $H$ is innately transitive of ASQ type.