Commit d5ec1748 authored by Phoebe Buckheister's avatar Phoebe Buckheister

meta: expect correct owner node ID for lookup-intent stat

when two clients race to create the same file in a meta-mirrored setup
only one of them can succeed. the other will have to either fail
entirely (for exclusive creates) or use the file created by the winning
client (for non-exclusive creates).

creates are usually accompanied by stat to retrieve important file
metadata as well. the losing client will see that the file has already
been created and only attempt to stat the file it has just found.
during stat it expects the file to be on the wrong node: owner checks
are done expecting primary-of-group(localNodeID) instead of expecting
localGroupID. for groups created with the id of their primary server
this usually succeeds (until failover happens), on others it will fail.

fixes #948

(cherry picked from commit ed46b0cc5e7916632b83cd67e8506573c5b20f0f)
parent 3deca9fa
......@@ -301,8 +301,7 @@ FhgfsOpsErr LookupIntentMsgEx::stat(EntryInfo* entryInfo, bool loadFromDisk, Sta
// check if we can stat on this machine or if entry is owned by another server
NumNodeID expectedOwner;
if (entryInfo->getIsBuddyMirrored())
expectedOwner = NumNodeID(
metaBuddyGroupMapper->getPrimaryTargetID(localNode.getNumID().val() ) );
expectedOwner = NumNodeID(metaBuddyGroupMapper->getLocalGroupID());
expectedOwner = localNode.getNumID();
#include <errno.h>
#include <fcntl.h>
#include <pthread.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
int waiting;
struct data {
const char* filename;
int result;
void* wait_and_open(void* data) {
struct data* args = data;
errno = 0;
while (__sync_fetch_and_add(&waiting, 0))
int fd = open(args->filename, O_CREAT | O_RDWR, 0644);
args->result = errno;
return NULL;
int main(int argc, char** argv) {
if (argc != 3)
return 1;
// this test does not *always* fail, even if the bug is present.
// repeat the test 10 times for a high enough chance to trigger.
for (int round = 0; round < 10; round++) {
struct data t1 = { argv[1], 0 };
struct data t2 = { argv[2], 0 };
pthread_t pt1, pt2;
waiting = 1;
pthread_create(&pt1, NULL, wait_and_open, &t1);
pthread_create(&pt2, NULL, wait_and_open, &t2);
usleep(100 * 1000);
__sync_fetch_and_and(&waiting, 0);
pthread_join(pt1, NULL);
pthread_join(pt2, NULL);
if (t1.result || t2.result) {
printf("id 1 errno %s\n", strerror(t1.result));
printf("id 2 errno %s\n", strerror(t2.result));
return 1;
return 0;
mount0 =["client0"]["mount"]
mount1 =["client1"]["mount"]
test = caller_locations(0, 1)[0].path.gsub /.pq$/, ".c"
on node do; tempdir do;
upload local: test
shell "gcc -std=gnu99 -pthread #{test.split("/").last}"
shell "./a.out #{mount0}/f #{mount1}/f"
end; end
- mgmtd
- meta
- meta
- storage
- helperd
- client
- client
- id: 1000
primary: 1
secondary: 2
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment