Merged 'mysql-5.7.13' into ps-5.7.13-merge

Merge blueprint: https://blueprints.launchpad.net/percona-server/+spec/merge-5.7.13

***
Conflict in 'libmysql/libmysql.ver.in' resolved in combining
'libmysqlclient' -> 'libperconaserverclient' rename with splitting
'@.0' and '@.3' libraries.

***
Conflict in 'sql/rpl_slave.cc' in 'global_init_info()' resolved in combining
both changes - call to 'fill_mts_gaps_and_recover()' put under binlog protection.

PS bugs:
lp:1358836 "Backup locks do not guarantee consistent SHOW SLAVE STATUS with binary log disabled"
(https://bugs.launchpad.net/percona-server/+bug/1358836)
lp:1372806 "Deadlock with LOCK BINLOG FOR BACKUP + SHOW SLAVE STATUS with binary log enabled"
(https://bugs.launchpad.net/percona-server/+bug/1372806)
(commit aaf2f1f)

Oracle bug:
(http://bugs.mysql.com/bug.php?id=77496)
(commit 3bffd42)

***
Reverted our fix for lp:1566642
"Replication does not work when @@GLOBAL.SERVER_UUID is missing on the master"
(https://bugs.launchpad.net/percona-server/+bug/1566642)
(commit 4663069) in favor of the upstream fix for
(http://bugs.mysql.com/bug.php?id=80962)
(commit 71c0780)

Removed our MTR test case "rpl.bug80962" as new checks from
"rpl.rpl_get_master_version_and_clock" completely covers it.

***
Reverted our fix for lp:1552682
"Audit log worker thread may crash on write call writing fewer bytes than requested"
(https://bugs.launchpad.net/percona-server/+bug/1552682)
(commit 35fa3b9) in favor of the upstream fix for
(commit 4b9d594)

***
Reverted our fix for lp:1454441
"Hanging 'System Lock' when executing 'flush table ... for export'"
(https://bugs.launchpad.net/percona-server/+bug/1454441)
(commit 5692683) in favor of the upstream fix for
(http://bugs.mysql.com/bug.php?id=77011)
(commit 4b714c4, 5,6->5.7 merge commit eff4a3c)

Removed our MTR test case "innodb.percona_bug1454441" as "innodb.flush-hang"
completely covers it.

***
Conflicting changes in 'log_buffer_extend()' and 'log_init()'
in 'storage/innobase/log/log0log.cc' combined into
"ut_zalloc_nokey(log_sys->buf_size * 2 + srv_log_write_ahead_size));"

Alignment assertions in 'log_buffer_switch()' changed to
"ut_ad(log_sys->buf == ut_align(log_sys->buf_ptr, srv_log_write_ahead_size));"

References:
lp:1578516 "Innodb crash recovery with ALL_O_DIRECT in 5.7 errno != EINVAL"
(https://bugs.launchpad.net/percona-server/+bug/1578516)
(commit 3eeb4a1)

(commit aa47808)

***
Removed MTR test case 'main.bug73556' as it tests the same Bug #73556 as
'main.percona_bug1354011'.

***
Re-recorded the following MTR tests after WL#9124
"Extension of inlined JSON path operato." (commit a5d4e7c) upstream implementation:
'main.opt_hints_pfs'
'perfschema.digest_table_full'
'perfschema.rpl_gtid_func'
'perfschema.start_server_low_digest'
'perfschema.start_server_low_digest_sql_length'
'perfschema.statement_digest'
'perfschema.statement_digest_consumers'
'perfschema.statement_digest_long_query'
'query_rewrite_plugins.digest_collision'
'query_rewrite_plugins.optional_columns'
'query_rewrite_plugins.verbose'

***
Re-recorded 'audit_log.audit_log_threadpool' as because of the
WL#5705 "Audit log plugin fine-grained pre-filtering" (commit 1edc52a)
a new 'MYSQL_AUDIT_CONNECTION_DISCONNECT' event is generated.

***
'rpl.percona_bug1331586' MTR test case re-enabled for valgrind as the root cause
why it was disabled originally was fixed in 5.7.13 Bug #22110916 / #78999
"INJECTED QUERY LOG EVENTS DO NOT SET EXPLICIT_DEFAULTS_TS, READ IT ON APPLYING"
(http://bugs.mysql.com/bug.php?id=78999)
(commit 626a9cd)

***
'rpl.rpl_checksum_cache' MTR test case re-enable for ASan as the root cause why it
was disabled originally was fixed in 5.7.13 Bug #22109863 / #78995
"Binlog_sender::read_event may access freed event buffer"
(https://bugs.mysql.com/bug.php?id=78995)
(commit 72f463e)

***
Removed extra "lldb" help string in 'mysql-test/mysql-test-run.pl'

***
Fixed SIGSEGV problem in 'pfs_instr-oom' unit test. This is a Percona fix for
Bug#81868 "pfs_instr-oom unit test fails on MySQL Server 5.7.13"
(http://bugs.mysql.com/bug.php?id=81868).

**
Disabled 'scalability_metrics.scalability_metrics' MTR test case as because of
the existing accessing freed memory problem lp:1441139
"handle_fatal_signal (sig=11) in Queue<PROF_MEASUREMENT>::pop | sql/sql_profile.h:127 | handle_fatal_signal (sig=11) in my_lfind"
(https://bugs.launchpad.net/percona-server/+bug/1441139)
and because of the additional MYSQL_AUDIT_CONNECTION_DISCONNECT event generated
in recently implemented WL#5705 "Audit log plugin fine-grained pre-filtering"
(commit 1edc52a), this test case cannot be run without crashing the server
anymore.

***
Percona-specific code containing '__attibute__(())' GCC extensions refactored to
utilize new 'MY_ATTRIBUTE()' macro.

***
Updated man pages from MySQL Server 5.7.13 source tarball.

***
VERSION raised to "5.7.13-6".
univ.i version raised to "6"

***
This merge also fixes lp:1541698 "Invalid InnoDB FTS Doc ID during INSERT"
(https://bugs.launchpad.net/percona-server/+bug/1541698) as
corresponding upstream Bug #80261 / #22679185
"Invalid InnoDB FTS Doc ID during INSERT"
(http://bugs.mysql.com/bug.php?id=80261)
was fixed in 5.7.13.

Author: percona-ysorokin

.gitignore

@@ -2841,7 +2841,6 @@ strings/test_decimal
 support-files/*.ini
 support-files/MacOSX/Description.plist
 support-files/MacOSX/Info.plist
-support-files/MacOSX/ReadMe.txt
 support-files/MacOSX/StartupParameters.plist
 support-files/MacOSX/postflight
 support-files/MacOSX/postinstall

VERSION

@@ -1,4 +1,4 @@
 MYSQL_VERSION_MAJOR=5
 MYSQL_VERSION_MINOR=7
-MYSQL_VERSION_PATCH=12
-MYSQL_VERSION_EXTRA=-5
+MYSQL_VERSION_PATCH=13
+MYSQL_VERSION_EXTRA=-6

client/base/abstract_program.cc

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2014, 2015  Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2014, 2016 Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -28,7 +28,7 @@ extern const char *load_default_groups[];
 
 my_bool Abstract_program::callback_option_parsed(
   int optid,
-  const struct my_option *opt __attribute__((unused)),
+  const struct my_option *opt MY_ATTRIBUTE((unused)),
   char *argument)
 {
   // Check if option uses My::Tools::Base::Options, and it should.

client/base/abstract_program.h

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2014, 2015 Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2014, 2016 Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -83,6 +83,10 @@ class Abstract_program : public Options::Composite_options_provider
    Prints program invocation message.
   */
   virtual void short_usage()= 0;
+  /**
+   Return error code
+  */
+  virtual int get_error_code()= 0;
 
 protected:
   Abstract_program();
@@ -108,7 +112,7 @@ class Abstract_program : public Options::Composite_options_provider
     If we have anonymous functions or binding this should be removed.
   */
   static my_bool callback_option_parsed(int optid,
-    const struct my_option *opt __attribute__((unused)),
+    const struct my_option *opt MY_ATTRIBUTE((unused)),
     char *argument);
 
   Options::Debug_options m_debug_options;

client/base/debug_options.cc

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2014, Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2014, 2016, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -72,7 +72,7 @@ void Debug_options::create_options()
 }
 
 
-void Debug_options::debug_option_callback(char *argument __attribute__((unused)))
+void Debug_options::debug_option_callback(char *argument MY_ATTRIBUTE((unused)))
 {
   if (this->m_dbug_option.has_value())
   {

client/base/debug_options.h

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2014, 2015 Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2014, 2016, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -52,7 +52,7 @@ class Debug_options : public Abstract_options_provider
 
 private:
 
-  void debug_option_callback(char *argument __attribute__((unused)));
+  void debug_option_callback(char *argument MY_ATTRIBUTE((unused)));
 
   Abstract_program* m_program;
   bool m_debug_info_flag;

client/base/help_options.cc

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2014, 2015 Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2014, 2016, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -46,13 +46,13 @@ void Help_options::create_options()
     this, &Help_options::version_callback));
 }
 
-void Help_options::help_callback(char* argument __attribute__((unused)))
+void Help_options::help_callback(char* argument MY_ATTRIBUTE((unused)))
 {
   this->print_usage();
   exit(0);
 }
 
-void Help_options::version_callback(char* argument __attribute__((unused)))
+void Help_options::version_callback(char* argument MY_ATTRIBUTE((unused)))
 {
   this->print_version_line();
   exit(0);

client/base/help_options.h

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2014, 2015, Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2014, 2016, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -51,8 +51,8 @@ class Help_options : public Abstract_options_provider
   virtual void print_usage();
 
 private:
-  void help_callback(char* argument __attribute__((unused)));
-  void version_callback(char* argument __attribute__((unused)));
+  void help_callback(char* argument MY_ATTRIBUTE((unused)));
+  void version_callback(char* argument MY_ATTRIBUTE((unused)));
 
   void print_version_line();
 

client/base/mysql_connection_options.cc

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2014, 2015, Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2014, 2016, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -50,19 +50,6 @@ Mysql_connection_options::Mysql_connection_options(Abstract_program *program)
   this->add_provider(&this->m_ssl_options_provider);
 }
 
-Mysql_connection_options::~Mysql_connection_options()
-{
-  my_boost::mutex::scoped_lock lock(m_connection_mutex);
-  for (vector<MYSQL*>::iterator it= this->m_allocated_connections.begin();
-    it != this->m_allocated_connections.end(); it++)
-  {
-    if (*it)
-    {
-      mysql_close(*it);
-    }
-  }
-}
-
 void Mysql_connection_options::create_options()
 {
   this->create_new_option(&this->m_bind_addr, "bind-address",
@@ -129,13 +116,7 @@ void Mysql_connection_options::create_options()
 
 MYSQL* Mysql_connection_options::create_connection()
 {
-  MYSQL *connection = new MYSQL;
-
-  {
-  my_boost::mutex::scoped_lock lock(m_connection_mutex);
-  this->m_allocated_connections.push_back(connection);
-  }
-  mysql_init(connection);
+  MYSQL *connection= mysql_init(NULL);
   if (this->m_compress)
     mysql_options(connection, MYSQL_OPT_COMPRESS, NullS);
 
@@ -184,6 +165,7 @@ MYSQL* Mysql_connection_options::create_connection()
     this->get_null_or_string(this->m_mysql_unix_port), 0))
   {
     this->db_error(connection, "while connecting to the MySQL server");
+    mysql_close(connection);
     return NULL;
   }
 
@@ -218,22 +200,22 @@ const char* Mysql_connection_options::get_null_or_string(
 
 #ifdef _WIN32
 void Mysql_connection_options::pipe_protocol_callback(
-  char* not_used __attribute__((unused)))
+  char* not_used MY_ATTRIBUTE((unused)))
 {
   this->m_protocol= MYSQL_PROTOCOL_PIPE;
 }
 #endif
 
 void Mysql_connection_options::protocol_callback(
-  char* not_used __attribute__((unused)))
+  char* not_used MY_ATTRIBUTE((unused)))
 {
   this->m_protocol=
     find_type_or_exit(this->m_protocol_string.value().c_str(),
     &sql_protocol_typelib, "protocol");
 }
 
 void Mysql_connection_options::secure_auth_callback(
-  char* not_used __attribute__((unused)))
+  char* not_used MY_ATTRIBUTE((unused)))
 {
   /* --secure-auth is a zombie option. */
   if (!this->m_secure_auth)

client/base/mysql_connection_options.h

@@ -74,7 +74,6 @@ class Mysql_connection_options
     @param program Pointer to main program class.
    */
   Mysql_connection_options(Abstract_program *program);
-  ~Mysql_connection_options();
 
   /**
     Creates all options that will be provided.
@@ -111,19 +110,13 @@ class Mysql_connection_options
    */
   void db_error(MYSQL* connection, const char* when);
 #ifdef _WIN32
-  void pipe_protocol_callback(char* not_used __attribute__((unused)));
+  void pipe_protocol_callback(char* not_used MY_ATTRIBUTE((unused)));
 #endif
-  void protocol_callback(char* not_used __attribute__((unused)));
-  void secure_auth_callback(char* argument __attribute__((unused)));
+  void protocol_callback(char* not_used MY_ATTRIBUTE((unused)));
+  void secure_auth_callback(char* argument MY_ATTRIBUTE((unused)));
 
   static bool mysql_inited;
 
-  /*
-   List of created connections. As we don't have memory management for
-   C structs we must clear it by options provider destruction.
-   */
-  std::vector<MYSQL*> m_allocated_connections;
-  my_boost::mutex m_connection_mutex;
   Ssl_options m_ssl_options_provider;
   Abstract_program *m_program;
   Nullable<std::string> m_protocol_string;

client/base/mysql_query_runner.cc

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2014, 2015, Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2014, 2016, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -28,17 +28,40 @@ using std::string;
 
 Mysql_query_runner::Mysql_query_runner(MYSQL* connection)
   : m_is_processing(new my_boost::atomic<bool>(false)),
+  m_is_original_runner(true),
   m_connection(connection)
 {}
 
 Mysql_query_runner::Mysql_query_runner(const Mysql_query_runner& source)
   : m_result_callbacks(source.m_result_callbacks),
   m_message_callbacks(source.m_message_callbacks),
   m_is_processing(source.m_is_processing),
+  m_is_original_runner(false),
   m_connection(source.m_connection)
 {
 }
 
+Mysql_query_runner::~Mysql_query_runner()
+{
+  if (m_is_original_runner)
+  {
+    for (std::vector<I_callable<int64, const Row&>*>::iterator
+      it= m_result_callbacks.begin(); it != m_result_callbacks.end(); ++it)
+    {
+      delete *it;
+    }
+    for (std::vector<I_callable<int64, const Message_data&>*>::iterator
+      it= m_message_callbacks.begin(); it != m_message_callbacks.end(); ++it)
+    {
+      delete *it;
+    }
+
+    delete m_is_processing;
+
+    mysql_close(this->m_connection);
+  }
+}
+
 Mysql_query_runner& Mysql_query_runner::add_result_callback(
   I_callable<int64, const Row& >* result_callback)
 {
@@ -280,7 +303,7 @@ void Mysql_query_runner::append_escape_string(
 
   int length = mysql_real_escape_string_quote(
     m_connection, &((*destination_string)[0]) + start_lenght, original,
-    original_length, '"');
+    (ulong)original_length, '"');
   destination_string->resize(start_lenght + length);
 }
 

client/base/mysql_query_runner.h

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2014, 2015, Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2014, 2016, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -51,6 +51,8 @@ class Mysql_query_runner
     Copy constructor.
    */
   Mysql_query_runner(const Mysql_query_runner& source);
+
+  ~Mysql_query_runner();
   /**
     Adds new callback to be called on every result row of query.
     If callback return value other than 0 then query execution, passing
@@ -214,6 +216,12 @@ class Mysql_query_runner
    */
   my_boost::atomic<bool>* m_is_processing;
 
+  /**
+    Indicates if this is original runner or a copy. In case of original the
+    cleanup is performed on destruction.
+  */
+  bool m_is_original_runner;
+
   MYSQL* m_connection;
 };
 

client/base/password_option.cc

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2001, 2015, Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2001, 2016, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -61,6 +61,9 @@ void Password_option::password_callback(char* argument)
   }
   else
   {
-    *this->m_destination_value = Nullable<string>(::get_tty_password(NULL));
+    char *password= ::get_tty_password(NULL);
+    *this->m_destination_value = Nullable<string>(password);
+    my_free(password);
+
   }
 }

client/base/ssl_options.cc

@@ -78,7 +78,7 @@ void Mysql_connection_options::Ssl_options::create_options()
 
 
 void Mysql_connection_options::Ssl_options::ca_option_callback(
-  char *argument __attribute__((unused)))
+  char *argument MY_ATTRIBUTE((unused)))
 {
   if (!ssl_mode_set_explicitly)
     ::opt_ssl_mode= SSL_MODE_VERIFY_CA;
@@ -101,7 +101,7 @@ void Mysql_connection_options::Ssl_options::apply_for_connection(
 
 
 void Mysql_connection_options::Ssl_options::use_ssl_option_callback(
-  char *argument __attribute__((unused)))
+  char *argument MY_ATTRIBUTE((unused)))
 {
   CLIENT_WARN_DEPRECATED("--ssl", "--ssl-mode");
   if (!opt_use_ssl_arg)
@@ -112,7 +112,7 @@ void Mysql_connection_options::Ssl_options::use_ssl_option_callback(
 
 
 void Mysql_connection_options::Ssl_options::ssl_verify_server_cert_callback(
-  char *argument __attribute__((unused)))
+  char *argument MY_ATTRIBUTE((unused)))
 {
   CLIENT_WARN_DEPRECATED("--ssl-verify-server-cert",
                          "--ssl-mode=VERIFY_IDENTITY");

client/check/mysqlcheck.cc

@@ -1,5 +1,5 @@
 /*
-   Copyright (c) 2001, 2015, Oracle and/or its affiliates. All rights reserved.
+   Copyright (c) 2001, 2016, Oracle and/or its affiliates. All rights reserved.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -274,7 +274,7 @@ static void usage(void)
 
 
 static my_bool
-get_one_option(int optid, const struct my_option *opt __attribute__((unused)),
+get_one_option(int optid, const struct my_option *opt MY_ATTRIBUTE((unused)),
 	       char *argument)
 {
   int orig_what_to_do= what_to_do;