Medieval society had law and order just like today. You couldn’t just march into your neighbour’s estates and take his castle by force - you’d become an outlaw.
Examples:
The Hundred Years War (France vs England)
Dispute between the Capetians and Angevins over who had the best claim to the Crown of France.
Wars of the Roses (York vs Lancaster)
Dispute between two cadet branches of the Angevins over who had the best right to succeed the extinct senior branch on the throne of England.
Battle of Nibley Green (between a couple of minor nobles)
Dispute between the Talbots and Berkeleys over who had the right to succeed to the Lisle inheritance.
This continued up till the modern era, with the War of the Spanish succession and so forth. Before the French Revolution, this is what wars tended to be about. Whether its about claiming an entire kingdom, or just a few local manors, people fought over exactly the same thing on the scale of their social status.
Taking possession of important estates, therefore, was much more about physically eliminating rival bloodlines than besieging castles.