Martial systems develop in response to their environments and the culture of the time. For fantasy settings, those environments include giant monsters, magic, and other general weirdness that would absolutely affect all martial systems everywhere in the setting. Unless your setting has only humans, or only humanoids with a similar height, reach, number of limbs, etc., your fantasy martial systems are going to have radically different considerations.
Any of these martial arts can be taken in place of a language for Monks, Fighters, Barbarians, Paladins, and Rangers. Or maybe as a Feat, GLOG Template, what have you.
A specialized form of underwater striking and grappling, which turn out to be a little different when gravity it a bit more subjective. Special attention is taken as to one's buoyancy, as the states of rising, neutral, and sinking often affect what limbs come in reach.
Being above your opponent is generally considered a winning position, as it allows you to keep your opponent from rising and accessing air, usually by kicking them in the face.
Practitioners receive no penalties when fighting underwater, and the amount of time they can hold their breath is doubled.
Novan Tunnel Fighting
Practiced in the drow volcanic city of November. Utilizes a braided training cord for grappling. Military uses use a sharp unbreakable cord that can slice through unarmored appendages with ease.
Techniques are divided into two categories: feet first, and head first. Feet first techniques emphasize pins and kicks while braced against a surface. Head first techniques make use of the braided cord to snag and drag limbs around sharp surfaces or garrote your opponent. Numerous pins have been developed to asphyxiate or otherwise incapacitate your opponent in confined spaces.
A sport version of this exists. An arena is constructed either vertically or horizontally, with one side composed of glass for spectators to see into. A rat maze is then constructed out of shaped stone and dirt. The fighters each start at one of multiple entrances and must find each other. This is done in total darkness. Those without darkvision will find learning the sport difficult.
Practitioners receive no penalties when grappling or using Light weapons when squeezing. Movement speed is 1/2 rather than 1/4th while squeezing.
Suthlands One-Armed Boxing
Numerous wars with the orcs of Shadowmountain have left many a veteran in the Suthlands missing eyes, noses, ears, and limbs. (The Shadowmountain orcs, contrary to the propaganda and scary stories, don't typically execute captives or kill opponents in battle. Instead they prefer to mark them by mutilation as a sign to other orcs that this person lost a fight, and hence might be below their effort.)
Missing arms are common. In this vacuum a fighting style was developed to make use of the lack of two arms: one-armed boxing. There's an emphasis on power and keeping the 'stump' protecting the head or body.
Practitioners deal more damage with a punch (but only with one arm), and only every other round of combat.
More martial art than fighting system. Makes use of quasi-religious meditation. The monk makes use of their mastery of their body in order to regenerate like a troll. Exemplary practitioners can even regrow limbs and rejuvenate aging body parts. Legendary practitioners can grow limbs and body parts that they didn't have originally, like a hydra sprouting new heads.
The mythical founder figure of the Troll Art was a woman named Fatoumata Din, and was described as to be so powerful in the practice that, with sufficient time and calories, could clone herself entirely.
Practitioners can use an action to gain temporary Regeneration. Available to monks only. No other adventuring professions has the introspection required.
Don't Eat Me
A self-defense system designed to assist the user in avoiding being predated by larger creatures. Applies to anything that could scoop you up and eat you whole: trolls, giants, kraken, giant birds.
Originally observed in small prey animals (to human), including mice, rats, and bugs. Rats would go on to be formal instructors in the school, opening academies to teach their (to them) giant neighbors.
First Rule of Don't Eat Me: If you can run, RUN!
Second Rule of Don't Eat Me: If you can hide, HIDE!
Third Rule of Don't Eat Me: Picking you up? Bite! Bite! Bite! Stab! Stab! Stab!
Fourth Rule of Don't Eat Me: If it drops you, scurry away!
Fifth Rule of Don't Eat Me: Out of options? Play dead! (Maybe it'll lose interest)
Sixth Rule of Don't Eat Me: Never travel alone. Always has at least three people (most big things have only two hands to grab).
Practitioners receive advantage to get out of grapples by larger creatures, and get an AC bonus to their attacks. Coincidentally, it also gives a minor boost to disarming techniques.
Armstrong's Long LAW Arms For Harm
For use with Armstrong's Long LAW Arms! What's the point of having a Long LAW Arm if you don't know how to use it? Having synthetic arms twice or three-times the length of somebody else's can be a big advantage! It can also be a liability if used improperly. Well, Armstrong's got the solution for you! A nifty fighting system to use with your superficially-extended limbs!
Armstrong advertises that the best use of the Long LAWs is to use them for slapping. Slapping really hard and fast, like a windmill arm falling from the heavens!
Practitioners ignore the penalties for using long arms. And they may or may not be taken seriously by other martial artists.
Makes use of body contortion and special paints to produce terrifying images. In this sense, the Glare Art is much more so art than any of the martial systems above. Derived from observance of petrifying creatures, such as the Basilisk and Medusa. Adherents will essentially practice making faces, until they can make a face that is truly terrifying enough to stop anything with sight in its tracks.
Being able to warp the face in this manner takes quite a lot of practice and skill, with varying levels of mastery present in the orders that teach them.
Initiates will begin by practicing spooky faces in a mirror, and will be taught how to formulate and apply the proper paints that emphasize certain features. Eventually, if they're good enough, they'll progress to higher stages by testing out their abilities on small plants.
Past a certain level of power, though, we pass beyond the realm of safe practice. Masters are capable of petrifying, truly petrifying someone with their glare alone. It takes a lot of energy and special knowledge of facial makeup to produce such an effect, but at a certain point it becomes outright dangerous to practice. There are several fables of overconfident masters using their powers wantonly, and all eventually catching glimpse of themselves by some fluke and self-petrifying.
Practitioners get advantages to intimidate their opponents. At higher levels they can Stun or even Petrify them at a glance. Available to monks only. No other adventuring professions has the introspection required.