German Chancellor Angela Merkel started laying the groundwork Monday for an unprecedented three-way governing coalition, but faced headwinds from conservative allies reeling from losses to an upstart nationalist party and clamoring for a tougher line on immigration and security.