Fenceless fencing, also called electronic fencing, hidden fencing or concealed fencing, has been used successfully with dogs, but it's less of a sure thing for cats.
These systems use a collar and a mild static electricity type sensation to discourage crossing over the buried boundaries. Be sure the fencing is guaranteed safe for cats, and even then think carefully. While the shock may not harm your cat, it may not work as well as it does for dogs, since cats respond differently to negative correction.
Too, your cat is not likely to stay inside the boundary despite the shock if a dog or dominant cat enters the property and gives chase. In addition, your cat is not safe from pet thieves, cruel people, predators and other dangers minimized by enclosures. This type of system is best used only when you can supervise your cat.