kernel - Fix pmap placemarker timing race

kernel - Fix pmap placemarker timing race

* Fix a timing race that could cause a thread to get stuck in
  "pvplw" indefinitely.  The timing window is very short and the
  race is fairly difficult to reproduce.

* For this race to occur, more than one interaction must take
  place on other cpus against the placemarker being waited on
  by pv_placemarker_wait().  It takes multiple interactions for
  the WAKEUP bit to be lost.

  The first interaction can clear the WAKEUP bit and issue a
  wakeup() before we manage to interlock.  The second interaction
  can then reserve the placemarker so our conditional fails and
  we tsleep().  The result is that we block forever.

  pv_placemarker_wait(pmap_t pmap, vm_pindex_t *pmark)
  {
          if (*pmark != PM_NOPLACEMARK) {
                  atomic_set_long(pmark, PM_PLACEMARK_WAKEUP);
                  tsleep_interlock(pmark, 0);
                  if (*pmark != PM_NOPLACEMARK)
                          tsleep(pmark, PINTERLOCKED, "pvplw", 0);
          }
  }

  Just moving the interlock to before setting the flag is not
  sufficient due to cuteness on my part in overloading the WAKEUP
  bit on top of NOPLACEMARK (NOPLACEMARK is '-1').

* The solution is to both properly order the interlock AND use
  a cmpset loop to prevent accidently setting the WAKEUP bit on
  a marker that has already been released.

	mark = *pmark;
	cpu_ccfence();
        while (mark != PM_NOPLACEMARK) {
                tsleep_interlock(pmark, 0);
                if (atomic_fcmpset_long(pmark, &mark,
                                       mark | PM_PLACEMARK_WAKEUP)) {
                        tsleep(pmark, PINTERLOCKED, "pvplw", 0);
                        break;
                }
        }

Reported-by: tuxillo