I'm running a drum brake like that, and it holds. I used Loctite green 620 on it for final assy. just in case, but it was working before that. The effort "before" that involved welding the adapter in place. That did NOT work out, so I wasted a drum with that experiment.
Did you see the kart plans drawn by and posted by "Dirtkarter" (Justin) on the diygokarts.com site? Your design looks exactly like the plans he posted in March 2014. I had been looking at them for awhile, thinking it would be a good kart to build.
Nice bends, nice welds. :thumbsup: