Its not an MOE, its what's referred to as a dense architecture, same as the Gemm...

		canyon289 5 months ago \| parent \| context \| favorite \| on: Gemma 3 270M: Compact model for hyper-efficient AI Its not an MOE, its what's referred to as a dense architecture, same as the Gemma3 models (But not 3n as noted)