First off, in order to make use of the powerful debugfs interface, it must be enabled within the kernel config. The relevant Kconfig macro is CONFIG_DEBUG_FS. Let's check whether it's enabled on our 5.4 custom kernel:
$ zcat /proc/config.gz | grep -w CONFIG_DEBUG_FS
Indeed it is; it's typically enabled by default in distributions.
Next, the default mount point of debugfs is /sys/kernel/debug. Thus, we can see that it is internally dependent on the sysfs kernel feature being present and mounted, which it is by default. Let's check where debugfs is mounted on our Ubuntu 18.04 x86_64 VM:
$ mount | grep -w debugfs
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
It is available and mounted at the expected location...