A peripheral is defined as an ancillary device that adds I/O or other functionality to a computer system. This can be anything from an I2C, SPI, or SD card controller to an audio or graphics device. Most of those are part of the physical SoC, with others added via interfaces that the SoC exposes to the outside world. Examples of external peripherals would be RAM (via the RAM controller) and a real-time clock (RTC).
One issue that one will likely encounter when using cheaper SBCs such as the Raspberry Pi, Orange Pi, and countless similar systems is that they usually lack an RTC, meaning that when they are powered off, they no longer keep track of the time. The thought behind this is usually that those boards will be connected to the internet anyway, so the OS can use an online time service (Network Time Protocol, or NTP) to synchronize the system time, thus saving board space.
One might end up using an SBC in a situation where no internet connection is available...