The Glendenning dukedom in this novel seems to function in an unorthodox way. Fortunately the laws that govern the peerage are unorthodox and there are as many exceptions as there are peerages. However, I think some of the exceptions here are unusual enough that they should be explained to the reader, who might otherwise be confused by what she interprets as an error.
1) The terms by which a title may be inherited can be dictated by a letter patent, in which the sovereign creates a dignity and specifies the rules by which it operates. Letters patent typically specify that a title may only be inherited directly -- passed from a father to the "heirs male of the body," i.e. his biological sons. The Glendenning dukedom is therefore atypical, though not unique, in that inheritance passes from father to grandson by way of a female (Daniel Lindenfell's mother).
A peerage generally becomes extinct when there is no one qualified to inherit it under the terms of its letter patent. However, there is at least one instance in which a monarch modified the terms of inheritance to include daughters when it became clear that the peer in question would not have any sons. Therefore it's plausible that the same thing might have happened in this case: George III (presumably) could have modified the terms of inheritance of the Glendenning dukedom so that, upon the death of the duke, his (presumably eldest) daughter would inherit the title and become duchess "in her own right." The dukedom would thereafter fall upon the heirs male of her body, i.e. Daniel Lindenfell. (The sovereign could also have altered the terms of the letter patent so that the dukedom would go to her son without falling on her in the interim; as Mrs. Lindenfell predeceases her father, however, it doesn't really matter in this case.)
2) However, even if the letter patent has been altered in this way, there can be no doubt as to who will inherit the dukedom. Several people may potentially be eligible for a title, but only one is eligible at any given time. This person, if not the eldest son of the current peer, is known as the heir presumptive and his position as front-runner can only be altered by the birth of a son (the heir apparent).
The rules of succession, however convoluted, are decisive and can only be changed by an official act on the part of the sovereign. Therefore, regardless of whether Peter or Daniel is the heir presumptive -- or, in other words, regardless of which of the duke's daughters is the conduit by which the dignity is passed from grandfather to grandson -- the order of succession would not be subject to interpretation or dispute. Nor can any amount of currying favor change it, unless it is with the Prince Regent himself and to such an extent that he officially alters the letter patent. The Duke of Glendenning himself has no authority over the grounds by which the title is held; all titles are gifts of the Crown, not so much owned by as on permanent loan to their holders.
3) The fact that the duke's title is separate from his property creates another problem toward the end of the book. Though the duke has no control over the inheritance of his title, he is free to dispose of his property however he likes; therefore, before the reading of the will Daniel can't assume that the house, or anything else that belonged to his grandfather, now belongs to him. The old duke could have left all or part of it to Peter Hathaway, his widow, or anyone else.
4) A duke's widow is only entitled to hold the title of Dowager Countess if the current duke is her direct descendent. As Jane never bore any children by the former duke, and as the title stands to be inherited either by Peter or by Daniel, she must lose the title of duchess as soon as her husband dies.
I drew a chart, too, but that was just for my own benefit. I get the sense that authors don't like charts.