Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Namenodes not been filtered by fs.defaultFS #320

Open
alanhe opened this issue Jun 5, 2023 · 0 comments
Open

Namenodes not been filtered by fs.defaultFS #320

alanhe opened this issue Jun 5, 2023 · 0 comments

Comments

@alanhe
Copy link

alanhe commented Jun 5, 2023

Steps to reproduce:
Add another cluster configuration in the testdata file: https://github.com/colinmarc/hdfs/blob/master/hadoopconf/testdata/conf/hdfs-site.xml#L37
to make it:

<configuration>
    <property>
        <name>dfs.nameservices</name>
        <value>tests</value>
    </property>
    <property>
       <name>dfs.ha.automatic-failover.enabled</name>
       <value>true</value>
    </property>
    <property>
        <name>dfs.ha.namenodes.mycluster</name>
        <value>nn1,nn2</value>
    </property>
    <property>
        <name>dfs.namenode.rpc-address.mycluster.nn1</name>
        <value>namenode1:8020</value>
    </property>
    <property>
        <name>dfs.namenode.rpc-address.mycluster.nn2</name>
        <value>namenode2:8020</value>
    </property>
    <property>
        <name>dfs.ha.namenodes.yourcluster</name>
        <value>nn1,nn2</value>
    </property>
    <property>
        <name>dfs.namenode.rpc-address.yourcluster.nn1</name>
        <value>namenode3:8020</value>
    </property>
    <property>
        <name>dfs.namenode.rpc-address.yourcluster.nn2</name>
        <value>namenode4:8020</value>
    </property>
</configuration>

Add a test case in hadoopconf_test.go

func TestDefaultFSAddress(t *testing.T) {
	conf, err := Load("testdata/conf")
	assert.NoError(t, err)

	addresses := conf.Namenodes()
	assert.Equal(t, 2, len(addresses))
}

Because the fs.defaultFS is hdfs://mycluster in testdata/conf/core-site.xml, I want to see 2 addresses been returned, but there are 4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant