I suspect the Pi can do considerably better, I haven't driven anything quite as big as 800x480 with 24bit images, however. You'd need something on the order of a 10Mhz bus speed to get 1fps if my maths isn't totally wonky. The Pi is capable of this ( some people use higher speeds to refresh APA102 pixels via the SPI bus ), I imagine it's something you'd want to implement in C: https://www.raspberrypi.org/documentation/hardware/raspberrypi/spi/README.md
Is there a specific reason for your display/controller/bus choice? The Pi would run rings around it outputting to an HDMI display or the official 7" Touchscreen.