To setup mesh system with G54 first you need to enable bridge mode on the modem and then you can connect mesh unit ax7800 to the modem first ethernet port to the router WAN port for setup. After enabling bridge mode on G54 it will be working as just a cable modem so you can use the surfboard central app to setup the mesh system. 

Since you have done with the SBC app setup for G54 first you need to remove the device on the app and perform hard reset on the modem after reset need to wait for the modem to get reboot completely and then you can login to the modem GUI using default login credentials to enable bridge mode and then you can install the surfboard central app for mesh system using the same login credentials.

http://arris.force.com/consumers/articles/General_FAQs/SURFboard-Central-Add-Remove-Device

http://arris.force.com/consumers/articles/General_FAQs/G54-Bridge-Mode-Setup

http://arris.force.com/consumers/articles/General_FAQs/mAX-W61-Wi-Fi-Cable-Modem-Internet-Setup