If a is the length of the edge of the cuboctahedron, the surface ist O=6*a²+8*(1/4)sqrt(3)a²=[6+2sqrt(3)]a².
You use the area formulas A=(1/4)sqrt(3)a² of the equilateral triangle and A=a² of the square.
The producing cube has the edge length sqrt(2)a.
Subtract the volumes of the eight triangle pyramids from the volume of the producing cube.
Volume of the pyramid
The formula is V=base*height/3. The triangle in front is chosen as the base.
There is V'=(a²/4)*[(1/2)sqrt(2)a]/3=(1/24)sqrt(2)a³.
Thus the volume of the cuboctahedron is V=2sqrt(2)a³-8V' = 2sqrt(2)a³-8(1/24)sqrt(2)a³ = (5/3)sqrt(2)a³.