Friday, November 20, 2015

'Copy to clipboard' button on web-page

Here are 3 ways to create such button:

Use flash-based ZeroClipboard - it's flash - I don't like it
Use clipboardData.SetData - works only for IE
Use modern w3c clipboard API (supported by modern browsers) - IMHO the best way

And here are my snippet for the last variant.

API: document.execCommand("copy") - returns true if successed
You can check if it enable by document.queryCommandEnabled("copy") - returns true if enable

Simplest case:
<html>
<body>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
<script LANGUAGE="JavaScript">
function ClipBoard(element) {
    var $temp = $("<textarea visibility: hidden;>");
    $("body").append($temp);
    $temp.val(element).select();
    document.execCommand("copy");
    $temp.remove();
}
</script>
<button onclick="ClipBoard('test\ntest')">Copy to clipboard</button>
</body>
</html>

I've needed next actions for click:

Ask data from server
Copy it to clipboard

But here are thing - w3c clipboard api works only for 'user-triggered thread' (and in the simplest using of ZeroClipboard it also doesn't work - maybe it need some another API for that - I don't know). And when you asking server by ajax - callback is another thread, not user-triggered. So, you can handle it next ways:

obsolete-but-working - not async ajax query
you can ask data in background and keep it in browser - in button click handler copy existing data. It's bad when you have a lot of data
you can ask user press button twice - first for loading, second for copying
you can in click handler ask data by ajax & show textarea with selected text - let user copy it by himself.

So, I choosed not the best by working - 1st case.

function copy_to_clipboard_button_handler()
{
    if (data == '')
    {
        var request_data = ...;

        $.ajax({
            type: "POST",
            url: this.url,
            data: request_data,
            async: false,

            success: function(data)
            {
                global_object.clipboard_data = data;
            }
        });

    }

    var $temp = $("<textarea visibility: hidden;>");
    $("body").append($temp);
    $temp.val(global_object.clipboard_data).select();
    document.execCommand("copy");
    $temp.remove();
}

So.

'async: false' means it will be user-triggered thread
usually in examples used <input> but it kills newline symbols
be careful with big amount of data - time for handling '$temp.val(global_object.clipboard_data).select()' grows exponential - on my desktop ~3MB handled ~20 seconds and ~30MB handled ~20 minutes and I didn't get patient to wait when it will finished. So, if you want copy more than several megabytes of data to clipboard - consider another variants - for example saving such amount of data to a file.

Monday, November 16, 2015

Mysql memo

Can't remember syntax of console mysql tools (bcs of rare using). Just put it here for myself:

dump database
"C:\Program Files\MySQL\MySQL Workbench 6.2 CE\mysqldump.exe" --host="%HOSTNAME%" --user=%USERNAME% --password=%PASSWORD% --result-file="out.txt" %DATABASE_NAME%

exec sql command
"C:\Program Files\MySQL\MySQL Workbench 6.2 CE\mysql.exe" -h %HOSTNAME% -u %USERNAME% --database=%DATABASE_NAME% --password=%PASSWORD% --execute="%QUERY" >out.txt

Using password in the command line string is insecure, so, possible to leave just '--password' without value. If not not to enter '--password', mysql think you want to connect without password and prints you an error.

Tuesday, November 10, 2015

Multilingual User Interface

Starting from Windows Vista there are appeared new mechanism - Multilingual User Interface.
If earlier you had one binary file where was code & strings, now it's another mechanism - you have 2 files: binary_filename (for example, crypt32.dll) + binary_filename.mui (for example, crypt32.dll.mui). Now binary file doesn't contain strings, and mui file have only one section - resources (name of section '.rsrc'), where in resource 'MUI' localize strings & their ID located. 'MUI' resource present and in binary with code - but there only a structure which helps find mui file.
MUI file located somewhere in C:\Windows\System32\en-US. en-US - strings in en-US languages, and every language pack presents own set of MUI files.
As I understood (didn't dig long & deeply, may be wrong), MUI file loaded by PE loader (by struct in 'MUI' resource in binary).
In binary with code to get string called LoadStringW() with id of this string.
So
What's good - logic & locale now divided, and you can switch locale dinamically.
What's bad - reversing of these binaries is hell now. If earlier you could find string in IDA, press 'x' and find usage - now you need go to MUI file, find ID, go to binary with code & search LoadStringW with this ID (and if you are lucky one - you will find it).

added

Just now I tried to start 'calc.exe', copied to another dir - it didn't started! with debugger I found out, that it failed in LoadStringW() - this func returned 0 and everything finished. So, I guessed - windows PE loader searched it into subdir - for example if I put calc.exe to c:\mydir\calc.exe, mui file searched by path c:\mydir\en-US\calc.exe.mui

Useful links

tiny article of MUI - [link] (note than 'FILEMUIINFO' struct - is not the same as in 'MUI' resource)
in comment here described struct in MUI resource - [link]

------------------------------------------------------------------------------------------
leave here are copy of that comment
------------------------------------------------------------------------------------------
The FILEMUIINFO structure is not equal to the MUI resource data. Further investigation of the resource data gives the following structure

// resource types defined by string only:
#define RT_MUI       TEXT("MUI")

// default resource name
#define RESFILE32_MUI_RESNAME    MAKEINTRESOURCE(1)

// mui signature
#define RESFILE32_MUI_SIGNATURE 0xFECDFECD

// mui version
#define RESFILE32_MUI_VERSION    0x00010000

// mui filetypes
#define RESFILE32_MUI_FILETYPE_MAIN 0x11
#define RESFILE32_MUI_FILETYPE_MUI   0x10

// ----
// notes about the checksum:
// ----
// - There are multiple methods given by MS about the calculation of the
//   checksums.
// - The actual case is there is no calculation of the checksum by the
//   resourceloader, it only compares checksums.
// - The checksums can be arbitary 16 byte codes.
// - The only requirement is that the MUI files have the same checksums in
//   their MUI resources as the base LN file.
// ----
// one incomplete method given by MS:
//   the main checksum is calculated from the major and minor version numbers of
//      a file and the file name (case sensitive), which are obtained from the
//      version resource
//   the service checksum is calculated based on the localizable resources
//      in the file
// ----
// another method given by MS:
//   The most common convention for checksum handling is to base the checksum on
//   the English (United States) resources. You are free to adopt a different
//   convention, as long as it is consistent for each LN file. It is also
//   acceptable to use the resource configuration file to assign an arbitrary
//   hexadecimal value of up to 16 hexadecimal digits as a checksum. It requires
//   adoption of a method using either GuidGen or some other tool to generate
//   checksum values.
// ----

// mui header
// - followed by array of bytes referenced by the header
// - every offset is 64-bit aligned, padding bytes are added accordingly
typedef struct
{
UINT32 Signature;         // equal to RESFILE32_MUI_SIGNATURE
UINT32 Size;              // size in bytes of MUI header and payload
UINT32 RcCfgVersion;      // equal to RESFILE32_MUI_VERSION
UINT32 Reserved1;         // equal to 0
UINT32 FileType;          // one of RESFILE32_MUI_FILETYPE_xxx
UINT32 Reserved2;         //              - SystemAttributes         ???
UINT32 Unknown;           // equal to 1   - UltimateFallbackLocation ???
BYTE   MainChecksum[16]; // checksum, see notes
BYTE   SvcChecksum[16];   // checksum, see notes
BYTE   Reserved3[24];
UINT32 OfsMainTypeNames; // offset to list with non-numerical resource types in main file (as multistring)
UINT32 SzeMainTypeNames; // size in bytes of MainTypeNames
UINT32 OfsMainTypeIDs;    // offset to list with numerical resource types in main file (as UINT32's)
UINT32 SzeMainTypeIDs;    // size in bytes of MainTypeIDs
UINT32 OfsMuiTypeNames;   // offset to list with non-numerical resource types in mui file (as multistring)
UINT32 SzeMuiTypeNames;   // size in bytes of MuiTypeNames
UINT32 OfsMuiTypeIDs;     // offset to list with numerical resource types in mui file (as UINT32's)
UINT32 SzeMuiTypeIDs;     // size in bytes of MuiTypeIDs
UINT32 Reserved4;
UINT32 Reserved5;
UINT32 OfsFbLanguage;     // offset to ultimate fallback language string
UINT32 SzeFbLanguage;     // size in bytes of FbLanguage, including eos

//
} TResFile32_MuiHdr;

Tuesday, August 25, 2015

Add a bit unix functional to windows server

Problem.

I have server with windows. I worked with this server through RDP. And sometime GUI subsystem stop to work correctly, but logs of my service showed that everything other worked fine - processes, services. So, I've needed an not-GUI mechanism to work with this server.

Action.

1) Install freesshd
If it asks something like "should I generate the keys?" - answer 'yes'
If it asks something like "install the service?" - answer 'yes'
2) Run freesshd as administrator
3) Add user&pass (password stored as sha1_hash), set tick 'shell' - it gives your user access to ssh
4) Go to 'control panel -> windows firewall -> advanced' and allow access from outside to 22 port
5) Exit from your gui instance of freesshd
6) install gnuwin327) add c:\gnuwin32 to system PATH (not default user PATH)
8) install midnight commander for Windows
9) add dir with installed mc.exe to system PATH (not default user PATH)
10) Restart service freesshd ('control panel -> administrative tools -> services')

So. Now you can connect to your windows server via ssh (for example, in Putty), use in shell comfortable gnu utils & midnight commander. If midnight commander looks bad, try to run it 'mc -ac'.

Tuesday, July 7, 2015

static Libxml++ hello_world in visual studio

So, how to build application, which uses libxml++.

For this moment I have directory 'D:\projects\libraries\libxmlplusplus' where libxml++ and all dependencies:

glib
glibmm
intl
libffi-3.2.1
libsigc++-2.4.1
libxml++-2.38.1
libxml2
pcre-8.37
zlib-1.2.8

So, I just made two property sheets - libxmlplusplus_x64_debug.props and libxmlplusplus_x64_release.props, and put them also into this directory.

libxmlplusplus_x64_debug.props:
Additional include directories:
    D:\projects\libraries\libxmlplusplus\libxml2\_build_x64_static_debug_mtd\include\libxml2
    D:\projects\libraries\libxmlplusplus\libxml++-2.38.1
    D:\projects\libraries\libxmlplusplus\glibmm\glib
    D:\projects\libraries\libxmlplusplus\glib\glib
    D:\projects\libraries\libxmlplusplus\glib
    D:\projects\libraries\libxmlplusplus\libxml++-2.38.1\MSVC_Net2010\libxml++
Additional library directories:
    D:\projects\libraries\libxmlplusplus\glibmm\MSVC_Net2010\Debug\x64\bin
    D:\projects\libraries\libxmlplusplus\libxml2\_build_x64_static_debug_mtd\lib
    D:\projects\libraries\libxmlplusplus\glib\build\win32\vs10\Debug\x64\bin
    D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_debug_mtd\.libs
    D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\MSVC_Net2010\x64\Debug
    D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_debug_mtd\Debug
    D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_debug_mtd\lib
    D:\projects\libraries\libxmlplusplus\libxml++-2.38.1\MSVC_Net2010\libxml++\x64\Debug
input:
    glibmm.lib
    libxml2_a.lib
    ws2_32.lib
    glib.lib
    gobject.lib
    libffi_convenience.lib
    winmm.lib
    sigc-vc100-d-2_0.lib
    pcred.lib;intl.lib
    xml++-vc100-d-2_6.lib

libxmlplusplus_x64_release.props:
Additional include directories:
    D:\projects\libraries\libxmlplusplus\libxml2\_build_x64_static_release_mt\include\libxml2
    D:\projects\libraries\libxmlplusplus\libxml++-2.38.1
    D:\projects\libraries\libxmlplusplus\glibmm\glib
    D:\projects\libraries\libxmlplusplus\glib\glib
    D:\projects\libraries\libxmlplusplus\glib
    D:\projects\libraries\libxmlplusplus\libxml++-2.38.1\MSVC_Net2010\libxml++
Additional library directories:
    D:\projects\libraries\libxmlplusplus\glibmm\MSVC_Net2010\Release\x64\bin
    D:\projects\libraries\libxmlplusplus\libxml2\_build_x64_static_release_mt\lib
    D:\projects\libraries\libxmlplusplus\glib\build\win32\vs10\Release\x64\bin
    D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_release_mt\.libs
    D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\MSVC_Net2010\x64\Release
    D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_release_mt\Release
    D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_release_mt\lib
    D:\projects\libraries\libxmlplusplus\libxml++-2.38.1\MSVC_Net2010\libxml++\x64\Release
input:
    glibmm.lib
    libxml2_a.lib
    ws2_32.lib
    glib.lib
    gobject.lib
    libffi_convenience.lib
    winmm.lib
    sigc-vc100-2_0.lib
    pcre.lib
    intl.lib
    xml++-vc100-2_6.lib

To check if everything works I've took sources from this article, cut off curl stuff and got something like that:

#include <libxml/tree.h>
#include <libxml/HTMLparser.h>
#include <libxml++/libxml++.h>

#include <iostream>
#include <string>

int main() {

    // Collect response
    std::string re = "bla bla fucking bla";

    // Parse HTML and create a DOM tree
    xmlDoc* doc = htmlReadDoc((xmlChar*)re.c_str(), NULL, NULL, HTML_PARSE_RECOVER | HTML_PARSE_NOERROR | HTML_PARSE_NOWARNING);

    // Encapsulate raw libxml document in a libxml++ wrapper
    xmlNode* r = xmlDocGetRootElement(doc);
    xmlpp::Element* root = new xmlpp::Element(r);

    // Grab the IP address
    std::string xpath = "//*[@id=\"locator\"]/p[1]/b/font/text()";
    auto elements = root->find(xpath);
    if (!elements.empty())
    {
        std::cout << "Your IP address is:" << std::endl;
        std::cout << dynamic_cast<xmlpp::ContentNode*>(elements[0])->get_content() << std::endl;
    }

    delete root;
    xmlFreeDoc(doc);

    return 0;
}

And what you need to build it:

Create project in visual studio
Add file with aforementioned source code
Choose architecture x64 & mode (Debug/Release)
Set static runtime (/MT or /MTd)
Add aforementioned property sheets
Build & Run

And you even don't get any warnings if you did everything what I adviced. And your exe won't have any exports :)

Monday, July 6, 2015

building libxml++ by visual studio

Problem

I want to parse html as DOM - by xpath. The only adequate description found here - http://blog.laplante.io/2014/11/parsing-html-c-revisited/, and it looks like libxml++ the only variant. But here are one huge problem - I'm developing in Visual Studio on Windows, and I prefer static libraries with static runtime. So, I was forced to build all dependencies.
I've found some blog - gtkmm-installation.blogspot.com - well, actuiall I've found it's github project - cloned it, tried to understand what the fuck is going on in this project - didn't understand how does it work and built everything by myself. Maybe if I've read these articles on blogspot I was able to understand it's github content, but somewhy I din't do it. Maybe in the future) Anyway, it's great experience & practice.
Every dependency building I've moved to special blog note, and this one if the final one about libxml++ building.

Foreword

I've made graph of dependencies with https://www.draw.io.

Tools & versions

visual studio 2013
git client
Compiled in visual studio 2013 libraries:

Compiled intl (version 20090911) in Visual Studio 2013 - [link]
Compiled pcre (version 8.37) in Visual Studio 2013 - [link]
Compiled libffi (version 3.2.1) in Visual Studio 2013 - [link]
Compiled zlib (version 1.2.8) in Visual Studio 2013 - [link]
Compiled libsig++ (version 2.4.1) in Visual Studio 2013 - [link]
Compiled glib (version 2.45.3) in Visual Studio 2013 - [link]
Compiled glibmm (version 2.45.3) in Visual Studio 2013 - [link]
Compiled libxml (version 2.9.2) in Visual Studio 2013 - [link]

Preparing and checking

---------------------------------------------------------
1) Build libxml2 (I think it's version 2.9.2)

    Date:            (03.07.2015 15:58:33)
    Commit hash:    73b2d73df8981e37a03dfdcf727d8bdafb019266

    You can use this tutorial:
        http://hostagebrain.blogspot.com/2015/06/building-libxml2-without-dependencies.html

    let's assume you have directories:
        D:\projects\libraries\libxmlplusplus\libxml2\_build_x64_static_release_mt
        D:\projects\libraries\libxmlplusplus\libxml2\_build_x64_static_debug_mtd

---------------------------------------------------------
2) Build zlib (version 1.2.8)

    You can use this tutorial:
        http://hostagebrain.blogspot.com/2015/06/note-of-mt-x64-static-zlib-compiling.html

    let's assume you have -MT here:
        D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_release_mt\include\zlib.h
        D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_debug_mtd\include\zconf.h
        D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_release_mt\lib\zlib.lib
        D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_release_mt\lib\zlib.pdb
    and -MTd here:
        D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_debug_mtd\include\zlib.h
        D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_debug_mtd\include\zconf.h
        D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_debug_mtd\lib\zlib.lib
        D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_debug_mtd\lib\zlib.pdb

---------------------------------------------------------
3) Build intl (version 20090911)

    You can use this tutorial:
        http://hostagebrain.blogspot.com/2015/06/proxy-libintl-note.html

    let's assume you have -MT here:
        D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_release_mt\include\libintl.h
        D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_release_mt\lib\intl.lib
    and -MTd here:
        D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_debug_mtd\include\libintl.h
        D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_debug_mtd\lib\intl.lib

---------------------------------------------------------
4) Build libffi (version 3.2.1)

    You can use this tutorial:
        http://hostagebrain.blogspot.com/2015/06/building-libffi-on-windows-by-visual.html

    let's assume you have -MT here:
        D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_release_mt\include
        D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_release_mt\.libs\libffi_convenience.lib
    and -MTd here:
        D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_debug_mtd\include
        D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_debug_mtd\.libs\libffi_convenience.lib

---------------------------------------------------------
5) Build pcre (version 8.37)

    You can use this tutorial:
        http://hostagebrain.blogspot.com/2015/07/building-pcre-on-visual-studio.html

    let's assume you have -MT here:
        D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_release_mt\include\pcre.h
        D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_release_mt\Release\pcre.lib
    and -MTd here:
        D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_debug_mtd\include\pcre.h
        D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_debug_mtd\Debug\pcred.lib

---------------------------------------------------------
6) libsigc++ (version 2.4.1)

    You can use this tutorial:
        http://hostagebrain.blogspot.com/2015/06/build-sigc-in-visual-studio.html

    let's assume you have -MT here:
        D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\MSVC_Net2010\x64\Release\sigc-vc100-2_0.lib
    and -MTd here:
        D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\MSVC_Net2010\x64\Debug\sigc-vc100-d-2_0.lib

---------------------------------------------------------
7) glib (I think it's version 2.45.3)

    Date:            (02.07.2015 14:06:17)
    Commit hash:    e337fe31637fe868ab94b488caf9b4898a2a1040

    You can use this tutorial:
        http://hostagebrain.blogspot.com/2015/06/build-minimal-glib-in-visual-studio.html

    lets assume you have projects here:
        D:\projects\libraries\libxmlplusplus\glib
    and binaries -MT here (despite we built Release_ExtPCRE, binaries will be in Release directory):
        D:\projects\libraries\libxmlplusplus\glib\build\win32\vs10\Release\x64\bin
    and binaries -MTd here (despite we built Debug_ExtPCRE, binaries will be in Debug directory):
        D:\projects\libraries\libxmlplusplus\glib\build\win32\vs10\Debug\x64\bin

---------------------------------------------------------
8) glibmm (2.45.3)

    Date:            (02.07.2015 22:03:00)
    Commit hash:    9c5e30622e38974e909fef60ea045d08000e33a0

    You can use this tutorial:
        http://hostagebrain.blogspot.com/2015/07/building-glibmm-by-visual-studio.html

    lets assume you have projects here:
        D:\projects\libraries\libxmlplusplus\glibmm
    and binaries -MT here
        D:\projects\libraries\libxmlplusplus\glibmm\MSVC_Net2010\Release\x64\bin
    and binaries -MTd here
        D:\projects\libraries\libxmlplusplus\glibmm\MSVC_Net2010\Debug\x64\bin

---------------------------------------------------------

Action

As always, I assume building is going on in directory 'D:\projects\libraries\libxmlplusplus' and all dependencies built by my instructions.

1) Take sources from http://ftp.gnome.org/pub/GNOME/sources/libxml++/

2) Open 'D:\projects\libraries\libxmlplusplus\libxml++-2.38.1\MSVC_Net2010\libxml++.sln'

3) Here are several files must be added in libxml++ project:

libxml++\validators\relaxngvalidator.cc
libxml++\validators\relaxngvalidator.h
libxml++\validators\schemavalidatorbase.cc
libxml++\validators\schemavalidatorbase.h
libxml++\validators\xsdvalidator.cc
libxml++\validators\xsdvalidator.h
libxml++\nodes\entitydeclaration.cc
libxml++\nodes\xincludeend.cc
libxml++\nodes\xincludestart.cc
libxml++\nodes\entitydeclaration.h
libxml++\nodes\xincludeend.h
libxml++\nodes\xincludestart.h
libxml++\attributedeclaration.cc
libxml++\attributenode.cc
libxml++\relaxngschema.cc
libxml++\schemabase.cc
libxml++\xsdschema.cc
libxml++\attributedeclaration.h
libxml++\attributenode.h
libxml++\relaxngschema.h
libxml++\schemabase.h
libxml++\xsdschema.h
libxml++\schema.cc
libxml++\schema.h

4) make libxml++ project static library

5) set /MT or /MTd to libxml++ project

6) make two property sheets - for 'Release x64' and for 'Debug x64'

Debug x64:

additional include directories

D:\projects\libraries\libxmlplusplus\glib
D:\projects\libraries\libxmlplusplus\glib\glib
D:\projects\libraries\libxmlplusplus\glibmm\glib
D:\projects\libraries\libxmlplusplus\libxml2\_build_x64_static_debug_mtd\include\libxml2
D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\_cygwin\include\sigc++-2.0
D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_debug_mtd\include
D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_debug_mtd\include
D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_debug_mtd\include

additional library directories

D:\projects\libraries\libxmlplusplus\glibmm\MSVC_Net2010\Debug\x64\bin
D:\projects\libraries\libxmlplusplus\libxml2\_build_x64_static_debug_mtd\lib
D:\projects\libraries\libxmlplusplus\glib\build\win32\vs10\Debug\x64\bin
D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_debug_mtd\.libs
D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\MSVC_Net2010\x64\Debug
D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_debug_mtd\Debug
D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_debug_mtd\lib

input

glibmm.lib
libxml2_a.lib
glib.lib
gobject.lib
sigc-vc100-d-2_0.lib
ws2_32.lib
pcred.lib
winmm.lib
libffi_convenience.lib
intl.lib

Release x64:

additional include directories

D:\projects\libraries\libxmlplusplus\glib
D:\projects\libraries\libxmlplusplus\glib\glib
D:\projects\libraries\libxmlplusplus\glibmm\glib
D:\projects\libraries\libxmlplusplus\libxml2\_build_x64_static_release_mt\include\libxml2
D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\_cygwin\include\sigc++-2.0
D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_release_mt\include
D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_release_mt\include
D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_release_mt\include

additional library directories

D:\projects\libraries\libxmlplusplus\glibmm\MSVC_Net2010\Release\x64\bin
D:\projects\libraries\libxmlplusplus\libxml2\_build_x64_static_release_mt\lib
D:\projects\libraries\libxmlplusplus\glib\build\win32\vs10\Release\x64\bin
D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_release_mt\.libs
D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\MSVC_Net2010\x64\Release
D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_release_mt\Release
D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_release_mt\lib

input

glibmm.lib
libxml2_a.lib
glib.lib
gobject.lib
sigc-vc100-2_0.lib
ws2_32.lib
pcre.lib
winmm.lib
libffi_convenience.lib
intl.lib

It must be exactly libxml2_a.lib, not libxml2.lib, becase 'libxml2.lib' makes dynamic linking.

7) Add property sheet to every project in solution.

8) Select all 'examples...' projects, go to their properties, and make two things:

delete .lib filename from 'input'
set runtime library - /MT or /MTd

9) To projects:

examples_dom_parse_entities
examples_dom_parser

add file 'testutilities.cc' to the project (it's located here: 'libxml++-2.38.1\examples\testutilities.cc').

8) Build all - everything must compiles successfully!

Run any example* .exe file - if everything is ok, it must just print something in console and correctly finishes.

building glibmm by visual studio

Problem

I've needed build glibmm (as dependency for libxml++), and wanted it as static x64 library with static runtime.

Tools & versions

glibmm (version near 2.45.3)
cygwin x32
visual studio 2013
git client
notepad++
far manager
Compiled in visual studio 2013 libraries:

Compiled intl (version 20090911) in Visual Studio 2013 - [link]
Compiled pcre (version 8.37) in Visual Studio 2013 - [link]
Compiled libffi (version 3.2.1) in Visual Studio 2013 - [link]
Compiled zlib (version 1.2.8) in Visual Studio 2013 - [link]
Compiled libsig++ (version 2.4.1) in Visual Studio 2013 - [link]
Compiled glib (version 2.45.3) in Visual Studio 2013 - [link]

glibmm:
Date: (02.07.2015 22:03:00)
Commit hash: 9c5e30622e38974e909fef60ea045d08000e33a0
just last commit on the moment of building

Action

I assume building take place in 'D:\projects\libraries\libxmlplusplus' and here are all dependencies already built by my instructions.

1) git clone https://github.com/GNOME/glibmm

2) delete '\r' in ./configure.ac & ./autogen.sh (I did it with notepad++)

3) Run 'Cygwin.bat' as administrator (it's important - somewhy it works only this way)

4) In cygwin console go to directory with sources and exec:
export GLIBMM_CFLAGS=-I"D:/projects/libraries/libxmlplusplus/libsigc++-2.4.1/_cygwin/include"
export GLIBMM_LIBS="D:/projects/libraries/libxmlplusplus/libsigc++-2.4.1/_cygwin/lib"
export GIOMM_CFLAGS=-I"D:/projects/libraries/libxmlplusplus/libsigc++-2.4.1/_cygwin/include"
export GIOMM_LIBS="D:/projects/libraries/libxmlplusplus/libsigc++-2.4.1/_cygwin/lib"
./autogen.sh
cd ./glib/src
make
cd ../../gio/src
make
These steps lead to generating a lot of source code files - which just absent without them.

5) Comment '# define GLIBMM_DLL 1' in files:

D:\projects\libraries\libxmlplusplus\glibmm\glib\glibmmconfig.h
D:\projects\libraries\libxmlplusplus\glibmm\MSVC_Net2010\glibmm\glibmmconfig.h

6) In visual studio project 'glibmm' authors forgot to add sources (which present in directory with sources) - you should add them to project:

binding.h
binding.cc

7) Remove all .lib filenames from property_sheet 'glibmmbuilddefinesprops'

8) Create two property sheets - for 'x64 debug' and for 'x64 release' and add them to 'glibmm' project.

x64 Debug:

additional include directories

D:\projects\libraries\libxmlplusplus\glib
D:\projects\libraries\libxmlplusplus\glib\glib
D:\projects\libraries\libxmlplusplus\glib\gmodule
D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\_cygwin\include\sigc++-2.0

additional library directories

D:\projects\libraries\libxmlplusplus\glib\build\win32\vs10\Debug\x64\bin

input

glib.lib
gobject.lib
gmodule.lib

x64 Release:

additional include directories

D:\projects\libraries\libxmlplusplus\glib
D:\projects\libraries\libxmlplusplus\glib\glib
D:\projects\libraries\libxmlplusplus\glib\gmodule
D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\_cygwin\include\sigc++-2.0

additional library directories

D:\projects\libraries\libxmlplusplus\glib\build\win32\vs10\Release\x64\bin

input

glib.lib
gobject.lib
gmodule.lib

9) set /MT or /MTd - to glibmm

10) set 'configuration type' -> 'static library' - to glibmm

11) build 'glibmm'

building PCRE on visual studio

Problem

Well, actually pcre library uses cmake & builds trivial. But I need some quick instruction for myself - so, let it be here - how to build pcre as static library with static runtime.

Action

1) Go to http://www.pcre.org/
find something like that:
'You can download the current releases of the PCRE and PCRE2 libraries from their official home via anonymous FTP:' ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/
Go to this FTP and take pcre sources, for example: pcre-8.37.zip
Download & unpack.

2) for debug version:
- open cmd.exe or far manager, go to directory with sources
- create directory '_build_x64_static_debug_mtd', go to it
- exec: cmake .. -G "Visual Studio 12 2013 Win64"
now in this directory will a lot of files, and one of them must be PCRE.sln
- open PCRE.sln
- select target architecture, mode, select 8 projects (which names is lowercase) in 'solution explorer' and set everywhere /MTd instead of /MDd
- build solution

2) for release version:
- open cmd.exe or far manager, go to directory with sources
- create directory '_build_x64_static_release_mt', go to it
- exec: cmake .. -G "Visual Studio 12 2013 Win64"
now in this directory will a lot of directories, and one of file must be PCRE.sln
- open PCRE.sln
- select target architecture, mode, select 8 projects (which names is lowercase) in 'solution explorer' and set everywhere /MT instead of /MD
- build solution

3) If you want use pcre ONLY as static library, you can add '#define PCRE_STATIC' into pcre.h before the first occurrence of the PCRE_STATIC.

4) I prefer move pcre.h to another directory, bcs when you include this header, extra files from this directory can be also included as side-effect, and lead to problems. So I put it like that:
pcre-8.37\_build_x64_static_release_mt\include\pcre.h
pcre-8.37\_build_x64_static_debug_mtd\include\pcre.h

Thursday, June 18, 2015

run idascript script to batch files

Problem

I had idapython script for extracting some data from file. And a lot of files, where from needed to extract these data.

Code

"C:\Program Files (x86)\IDA 6.8\idaw.exe" -Lfilename.log -A -S_try_script.py filename

it will open 'filename' in console ida and run script '_try_script.py'. In this script 'main' must be like that:

if __name__ == '__main__':
    idaapi.autoWait()
    my_analyze_func()
    Exit(0)

it waits to complete autoanalysis, executes script and exit. Ida console log will be in 'filename.log'.
So, you can use it from batch handling of the files.

self-unpacking archives from cmd

Problem

Recently I've needed to pack several files (exe+dll) into one executable, which after unpacking of content will run one of unpacked files. And do this with several bunches of files - so, console commands needed - not gui. It's some constant executable need to test with different dll

Tools

7z
7zS.sfx (I've found it in something like 7z-extra-922.7z)
Python

Action

First of all - you need to make file config.txt where you write program name, which must be run after unpacking (in my case I've needed to make dll think it's in IE, so my program name is 'iexplore.exe', but it's just executable, which loads 'current.dll' and waits)

;!@Install@!UTF-8!
Title="7-Zip 4.00"
Directory=""
RunProgram="%%T\\iexplore.exe"
;!@InstallEnd@!

it must be in UTF-8, so you can save it in UTF-8 from notepad.exe, for example.

and here are python script, where you can find all necessary tools & keys:

import shutil    # copy2
import os

filenames = [
'dll_00.dll',
'dll_01.dll',
'dll_02.dll',
]

def run():
    for filename in filenames:
        shutil.copy2(filename, './current.dll')
        os.system('"C:\\Program Files\\7-Zip\\7z.exe" a archive.7z ./iexplore.exe ./current.dll')

        destination = open(filename+'.sfx.exe', 'wb')
        shutil.copyfileobj(open(r"D:\tools\7z-extra-920\7zS.sfx", 'rb'), destination)
        shutil.copyfileobj(open(r"config.txt", 'rb'), destination)
        shutil.copyfileobj(open(r"archive.7z", 'rb'), destination)
        destination.close()

        os.system('sfx_handler '+filename+'.sfx.exe')
        os.remove('archive.7z')
        os.remove('current.dll')

if __name__ == '__main__':
    run()

build sigc++ in visual studio

Problem

I need to compile static x64 library sigc++ with static runtime in Visual Studio 2013. It's required to compile glibmm.

Tools and versions:

sigc++ 2.4.1 (will also note of 2.0.6 version)
visual studio 2013
cygwin (32-bits in my case)

Action

1. At http://libsigc.sourceforge.net/stable.shtml find link on ftp & download some (for today last version - 2.4.1).
2. open Cygwin.bat
3. go to dir with sigc++
4. ./configure
5. in 'sigc++config.h' comment '# define SIGC_DLL 1'it needs only if you want to build static library. __declspec(dllimport) from defines will fail building otherwise.
be attentive - there are as minimum two files, where you must to do it:
- libsigc++-2.4.1\sigc++config.h
- libsigc++-2.4.1\MSVC_Net2010\sigc++config.h
6. open 'D:\tmp\libraries\libsigc++-2.4.1\MSVC_Net2010\libsigc++2.sln'
7. in 'project settings -> general' of the project 'libsigc++' set 'static library'
8. set runtime library /MT or /MTd in:

all projects properties
in all files in project libsigc++
in property sheets 'Upgrade from VC 7.1', 'Microsoft.Cpp.x64.user'

9. build
here will built a lot of exe - no one must crashes

Note for those who builds it for glibmm

You need to compile it also with cygwin (it will be required by glibmm to generate some source files). So, open x32 cygwin (don't know if x64 cygwin will works - on some another projects it didn't work), go to directory with sources (for example it's 'd:/projects/libraries/libxmlplusplus/libsig++-2.4.1'), and exec:
./configure --prefix=d:/projects/libraries/libxmlplusplus/libsigc++-2.4.1/_cygwin
make
make install
and copy
D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\sigc++config.h
to
D:\projects\libraries\libxmlplusplus\libsigc++-2.4.1\_cygwin\include\sigc++-2.0\sigc++config.h
and don't forget check - '# define SIGC_DLL 1' must be commented.

Note of version 2.0.6.

After ./configure you need to make several addition steps:

add path to sigc++ library into 'additional include directories' in every test project
insert '#include <cstddef>' into the beginning of files: signal_base.h, signal.h
in 'test_deduce_result_type' replace "main()" to "int main()" & insert "return 0;" into the end of main function.
also maybe you will need to comment define 'SIGC_TYPEDEF_REDEFINE_ALLOWED'.
replace sig++config.h in root sig++ directory by file sig++config.h from MSVC_Net2003 directory.

Msys built 2.0.6 correctly.
Also interesting - in the internet people advices open solution file after ./configure & make, but I've found make don't change source files - so, you don't need to call 'make' - only './configure'.

build glib in Visual Studio

Problem.

I've needed to build glib (actually finally I want to get libxml++, and glib is one of dependencies). Will do it static library with static runtime (/MT and /MTd) x64. Firstly I wanted to build only part of glib, but finally I built whole library successfully.

Tools and versions (my case):

Visual Studio 2013
Python (I have python 2.7)
Perl (I have ActivePerl x64)
git client
x32 cygwin
far manager
notepad++
Compiled intl (version 20090911) in Visual Studio 2013 - [link]
Compiled pcre (version 8.37) in Visual Studio 2013 - [link]
Compiled libffi (version 3.2.1) in Visual Studio 2013 - [link]
Compiled zlib (version 1.2.8) in Visual Studio 2013 - [link]

glib version - it's near 2.45.3
Date: (02.07.2015 14:06:17)
Commit hash: e337fe31637fe868ab94b488caf9b4898a2a1040
just last commit from git for the moment of compilation.

Action

1. git clone git://git.gnome.org/glib

2. open x32 cygwin - Cygwin.bat

3. go to dir with glib sources

4. replace all '\r' to '' in files (I did it in notepad++):

autogen.sh
configure.ac

5. exec in cygwin:

./autogen.sh
cd ./gio
make

it must finishes with error, but it will create file 'gdbus-daemon-generated.c' - it's everything what we need.
Now we don't need cygwin - can close it.

6.exec python script (I did it with python 2.7), located in 'glib/build/win32':
python setup.py --perl=C:\Perl64\bin\perl.exe

it's like ./configure. For example, it copies config.h.win32 to config.h with replacing some substrings to real value (for example - @MAJOR_VERSION@ to concrete number) - without it visual studio can't find config.h. Also here are exists './glib/win32-fixup.pl', but you don't need touch this file - I think it's deprecated.
But it's not fully-functional configure, bcs it doesn't create file 'gdbus-daemon-generated.c', and we still need to exec autogen & make before it.

7. open ./glib/build/win32/vs10/glib.sln

8. choose mode x64 (I will consider modes Debug_ExtPCRE and Release_ExtPCRE)

9. in project properties, general -> change 'dynamic library' to 'static library' for projects:

gio
glib
gmodule
gobject
gthread

10. open 'property sheets' and delete all 'user macros' in sheet named 'glibgensrcsprops'

11. Create two property sheets - one for release x64, another for debug x64. I assume all 4 dependency libraries located in 'D:\projects\libraries\libxmlplusplus' and built by my instructions.

Release_ExtPCRE x64:

additional include directories

D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_release_mt\include
D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_release_mt\include
D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_release_mt\include
D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_release_mt\include

additional library directories

D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_release_mt\lib
D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_release_mt\Release
D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_release_mt\.libs
D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_release_mt\lib

input

ws2_32.lib
winmm.lib
libffi_convenience.lib
Iphlpapi.lib
Dnsapi.lib
zlib.lib
pcre.lib

Debug_ExtPCRE x64:

additional include directories

D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_debug_mtd\include
D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_debug_mtd\include
D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_debug_mtd\include
D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_debug_mtd\include

additional library directories

D:\projects\libraries\libxmlplusplus\intl\_build_x64_static_debug_mtd\lib
D:\projects\libraries\libxmlplusplus\pcre-8.37\_build_x64_static_debug_mtd\Debug
D:\projects\libraries\libxmlplusplus\libffi-3.2.1\_build_x64_static_debug_mtd\.libs
D:\projects\libraries\libxmlplusplus\zlib-1.2.8\_build_x64_static_debug_mtd\lib

input

ws2_32.lib
winmm.lib
libffi_convenience.lib
Iphlpapi.lib
Dnsapi.lib
zlib.lib
pcred.lib

add this property lists to every project in solution

12. Set static runtime (/MT and /MTd accordingly) for all projects except:

glib-install
gspawn-win32-helper
gspawn-win32-helper-console

glib-install - it's some strange thing - without code. gspawn-win32-helper and gspawn-win32-helper-console required some function (__wgetmainargs), which present only in dinamic library.

13. The goal of this step - get rid from dll export (if you don't do it - executables, which use libxml++ will have export O_o. So, you need make several replaces.
in 'glib\glib\gtypes.h':

'# define GLIB_VAR __declspec(dllexport)' -> '# define GLIB_VAR extern'
'# define GLIB_VAR extern __declspec(dllimport)' -> '# define GLIB_VAR extern'

in 'glib\config.h':

'#define _GLIB_EXTERN __declspec (dllexport) extern' -> '#define _GLIB_EXTERN extern'

in 'glib\gobject\gparamspecs.h' :

'# define GOBJECT_VAR __declspec(dllexport)' -> '# define GOBJECT_VAR extern'
# define GOBJECT_VAR extern __declspec(dllimport)' -> '# define GOBJECT_VAR extern'

14. find property sheet 'glibbuilddefinesprops' (for example, it's property sheet of 'glib-compile-resources' project), open 'user macros' and delete:
DllExportDefines    Dll_EXPORT

15. And finally one dirty hack) The point is - looks like glib usually used on Windows only as dynamic library. In dynamic libraries on Windows exists function DllMain, which in glib implemented to fill some internal structure with pointers to internal functions. When you build this library as static, there are no called DllMain, this structure is not initialized (filled by zeros), and glibmm has some global object, and everything crashes into constructor of this object, because it calles some glib function, and executed 'call' by address zero.
So, the point of my dirty hack is - initialize this internal structure in the first glib function, which called from glibmm.
so, in file glib -> glib_init.c

BOOL WINAPI DllMain (HINSTANCE hinstDLL,
                    DWORD     fdwReason,
                    LPVOID    lpvReserved);

HMODULE glib_dll;

BOOL WINAPI
DllMain (HINSTANCE hinstDLL,
        DWORD     fdwReason,
        LPVOID    lpvReserved)
{
switch (fdwReason)
    {
    case DLL_PROCESS_ATTACH:
    glib_dll = hinstDLL;
    g_clock_win32_init ();
#ifdef THREADS_WIN32
    g_thread_win32_init ();
#endif
    glib_init ();
    break;

    case DLL_THREAD_DETACH:
#ifdef THREADS_WIN32
    g_thread_win32_thread_detach ();
#endif
    break;

    default:
    /* do nothing */
    ;
    }

return TRUE;
}

replace by

HMODULE glib_dll;

int is_glib_initialized = 0;

void
glib_init_ctor (void)
{
    if (is_glib_initialized == 0)
    {
        glib_dll = 0;
        g_clock_win32_init ();
        g_thread_win32_init ();
        glib_init ();
        is_glib_initialized = 1;
    }
}

and in glib -> gthread-win32.c

void
g_mutex_init (GMutex *mutex)
{
g_thread_impl_vtable.InitializeSRWLock (mutex);
}

replace by

extern void glib_init_ctor(void);
void
g_mutex_init (GMutex *mutex)
{
glib_init_ctor();
g_thread_impl_vtable.InitializeSRWLock (mutex);
}

16. build all

unsuccessful can be only: gspawn-win32-helper and gspawn-win32-helper-console. It's okay.

Possible errors for googling people

problem - you didn't called 'setup.py'
-----------------------------------------------
1> Copying config.h from config.h.win32...
1> The system cannot find the file specified
-----------------------------------------------
what to do - step 2 in this post

Saturday, June 13, 2015

proxy-libintl note

proxy-libintl - it's tiny library for glib - used for it's internal purposes - as proxy library. It consists of 2 files: libintl.h and libintl.c. To build this library - just create 'static library' visual studio project (name it 'intl', bcs glib wait file intl.lib), add these files to project, set mode & settings (/MT & /MTd in my case) and just build it.
Download sources you can here.

Also you can copy symbols to the library (without it you will receive warnings later). For example, if you put lib file into subdirectories _build_x64_static_debug_mtd/lib and _build_x64_static_release_mt/lib, you can copy
intl\intl\x64\Debug\vc120.pdb
to
intl\_build_x64_static_debug_mtd\lib\vc120.pdb
and
intl\intl\x64\Release\vc120.pdb
to
intl\_build_x64_static_release_mt\lib\vc120.pdb

building libffi on windows by visual studio

problem

It's not trivial task - build libffi by visual studio.

tools & versions

What do you need to build libffi on windows:

Visual Studio (2013 in my case)
Cygwin 32-bits (I don't know why but cygwin64 doesn't work)

problem solving in a short for x64

0. Download and unpack (I'm using FAR manager for unpacking) libffi sources.
1. Run "VS2013 x64 Native Tools Command Prompt"
2. Open in it Cygwin.bat, for example:
    c:\cygwin\Cygwin.bat
3. Go to directory with sources, for example:
    cd d:/tmp/libraries/libffi-3.2.1
4. Make directory for building and go to it, for example:
    mkdir build-x64
    cd ./build-x64
5. Then you need to configure, and here you need decide which runtime do you want - dynamic (/MD) or static (/MT)
    dynamic (/MD): ../configure CC="../msvcc.sh -m64" CXX="../msvcc.sh -m64" LD=link CPP="cl -nologo -EP" --build=x86_64-unknown-cygwin
    static release (/MT): ../configure CC="../msvcc.sh -DUSE_STATIC_RTL -m64" CXX="../msvcc.sh -DUSE_STATIC_RTL -m64" LD=link CPP="cl -nologo -EP" --build=x86_64-unknown-cygwin
    static debug (/MTd): ../configure CC="../msvcc.sh -DUSE_STATIC_RTL -DUSE_DEBUG_RTL -m64" CXX="../msvcc.sh -DUSE_STATIC_RTL -DUSE_DEBUG_RTL -m64" LD=link CPP="cl -nologo -EP" --build=x86_64-unknown-cygwin
6. Then in ./include/ffitarget.h will be symlink (visual studio don't understand symlinks) - and this symlink has some strange access rights, so, for example in FAR manager (not in this cygwin console) do the next:
    - delete './include/ffitarget.h'
    - copy '../src/x86/ffitarget.h' to './include/ffitarget.h'
7. And in this ffitarget.h file you need to comment define '#define FFI_TARGET_HAS_COMPLEX_TYPE'
8. Using any text editor - in 'libffi-3.2.1\src\x86\win64.S' replace 'jmp SHORT' to 'jmp' under labels:
    'ret_uint16$:'
    'ret_sint16$:'
9. make
10. In subdirectory ./.libs you will get files.

problem solving in a short for x86

0. Download and unpack (I'm using FAR manager for unpacking) libffi sources.
1. Run "VS2013 x86 Native Tools Command Prompt"
2. Open in it Cygwin.bat, for example:
    c:\cygwin\Cygwin.bat
3. Go to directory with sources, for example:
    cd d:/tmp/libraries/libffi-3.2.1
4. Make directory for building and go to it, for example:
    mkdir build-x86
    cd ./build-x86
5. Then you need to configure, and here you need decide which runtime do you want - dynamic (/MD) or static (/MT)
    dynamic: .../configure CC="../msvcc.sh" CXX="../msvcc.sh" LD=link CPP="cl -nologo -EP" --build=i686-unknown-cygwin
    static: ../configure CC="../msvcc.sh -DUSE_STATIC_RTL" CXX="../msvcc.sh -DUSE_STATIC_RTL" LD=link CPP="cl -nologo -EP" --build=i686-unknown-cygwin
6. Then in ./include/ffitarget.h will be symlink, and visual studio don't understand symlinks - so, replace it by real file.    cp ../src/x86/ffitarget.h ./include
7. And in this ffitarget.h file you need to comment define '#define FFI_TARGET_HAS_COMPLEX_TYPE'
8. make
9. In subdirectory ./.libs you will get files.

useful sources and links

0. 'README' file in sources gives you couple advices, but it's not enough
1. This guy also gives good advices - [link], I've backuped it [here]

possible errors (for googling people)

1) if you open cygwin64\Cygwin.bat instead of cygwin\Cygwin.bat
    -----------------------------------------------------------------------------
    checking how to run the C++ preprocessor... /lib/cpp
    configure: error: in `/cygdrive/d/tmp/libraries/libffi-3.2.1/build-x64':
    configure: error: C++ preprocessor "/lib/cpp" fails sanity check
    See `config.log' for more details
    -----------------------------------------------------------------------------
    what to do - use 32-bits cygwin

2) if you set --build=i686... instead of --build=x86_64..., when you want x64 binaries
    -----------------------------------------------------------------------------
    ml -nologo -safeseh -c ./win32.asm
    ../msvcc.sh: line 241: ml: command not found
    Makefile:1338: ошибка выполнения рецепта для цели <src/x86/win32.lo>
    make[2]: *** [src/x86/win32.lo] Ошибка 1
    make[2]: выход из каталога </cygdrive/d/tmp/libraries/libffi-3.2.1/build-x64>
    Makefile:1596: ошибка выполнения рецепта для цели <all-recursive>
    make[1]: *** [all-recursive] Ошибка 1
    make[1]: выход из каталога </cygdrive/d/tmp/libraries/libffi-3.2.1/build-x64>
    Makefile:730: ошибка выполнения рецепта для цели <all>
    make: *** [all] Ошибка 2
    -----------------------------------------------------------------------------

    what to do - use --build=i686-unknown-cygwin

3) if you forgot replace symlink by real file
    -----------------------------------------------------------------------------
    include\ffitarget.h(1) : error C2059: syntax error : '!'
    C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\crtdefs.h(642) : error C2081: 'uintptr_t' : name in formal parameter list illegal
    C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\stddef.h(71) : error C2143: syntax error : missing '{' before '__cdecl'
    include\ffi.h(223) : error C2016: C requires that a struct or union has at least one member
    include\ffi.h(223) : error C2061: syntax error : identifier 'ffi_abi'
    include\ffi.h(232) : error C2059: syntax error : '}'
    include\ffi.h(240) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(240) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(241) : error C2146: syntax error : missing ';' before identifier 'abi'
    include\ffi.h(242) : error C2059: syntax error : 'type'
    include\ffi.h(246) : error C2059: syntax error : ')'
    include\ffi.h(263) : error C2016: C requires that a struct or union has at least one member
    include\ffi.h(263) : error C2061: syntax error : identifier 'ffi_sarg'
    include\ffi.h(264) : error C2061: syntax error : identifier 'uint'
    include\ffi.h(264) : error C2059: syntax error : ';'
    include\ffi.h(268) : error C2059: syntax error : '}'
    include\ffi.h(281) : error C2061: syntax error : identifier 'ffi_java_raw'
    include\ffi.h(281) : error C2059: syntax error : ';'
    include\ffi.h(285) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(285) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(286) : error C2059: syntax error : 'type'
    include\ffi.h(290) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(290) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(290) : error C2059: syntax error : 'type'
    include\ffi.h(290) : error C2059: syntax error : ')'
    include\ffi.h(291) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(291) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(291) : error C2143: syntax error : missing ';' before '*'
    include\ffi.h(291) : error C2059: syntax error : 'type'
    include\ffi.h(291) : error C2059: syntax error : ')'
    include\ffi.h(292) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(292) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(292) : error C2059: syntax error : ')'
    include\ffi.h(298) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(298) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(299) : error C2059: syntax error : 'type'
    include\ffi.h(303) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(303) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(303) : error C2059: syntax error : 'type'
    include\ffi.h(303) : error C2059: syntax error : ')'
    include\ffi.h(304) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(304) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(304) : error C2143: syntax error : missing ';' before '*'
    include\ffi.h(304) : error C2059: syntax error : 'type'
    include\ffi.h(304) : error C2059: syntax error : ')'
    include\ffi.h(305) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(305) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(305) : error C2059: syntax error : ')'
    include\ffi.h(433) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(433) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(434) : error C2146: syntax error : missing ';' before identifier 'abi'
    include\ffi.h(435) : error C2059: syntax error : 'type'
    include\ffi.h(437) : error C2059: syntax error : ')'
    include\ffi.h(439) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(439) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(440) : error C2146: syntax error : missing ';' before identifier 'abi'
    include\ffi.h(441) : error C2059: syntax error : 'type'
    include\ffi.h(444) : error C2059: syntax error : ')'
    include\ffi.h(446) : error C2143: syntax error : missing ')' before '*'
    include\ffi.h(446) : error C2143: syntax error : missing '{' before '*'
    include\ffi.h(447) : error C2059: syntax error : 'type'
    ../include\ffi_common.h(81) : error C2143: syntax error : missing ')' before '*'
    ../include\ffi_common.h(81) : error C2143: syntax error : missing '{' before '*'
    ../include\ffi_common.h(81) : error C2059: syntax error : ')'
    ../include\ffi_common.h(82) : error C2143: syntax error : missing ')' before '*'
    ../include\ffi_common.h(82) : error C2143: syntax error : missing '{' before '*'
    ../include\ffi_common.h(83) : error C2059: syntax error : 'type'
    ../include\ffi_common.h(83) : error C2059: syntax error : ')'
    ../include\ffi_common.h(88) : error C2016: C requires that a struct or union has at least one member
    ../include\ffi_common.h(88) : error C2061: syntax error : identifier 'ffi_cif'
    ../include\ffi_common.h(91) : error C2059: syntax error : '}'
    C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\stdlib.h(235) : error C2081: 'uintptr_t' : name in formal parameter list illegal
    ../src/prep_cif.c(107) : error C2143: syntax error : missing ')' before '*'
    ../src/prep_cif.c(107) : error C2143: syntax error : missing '{' before '*'
    ../src/prep_cif.c(107) : error C2146: syntax error : missing ';' before identifier 'abi'
    ../src/prep_cif.c(108) : error C2059: syntax error : 'type'
    ../src/prep_cif.c(111) : error C2059: syntax error : ')'
    ../src/prep_cif.c(226) : error C2143: syntax error : missing ')' before '*'
    ../src/prep_cif.c(226) : error C2143: syntax error : missing '{' before '*'
    ../src/prep_cif.c(226) : error C2146: syntax error : missing ';' before identifier 'abi'
    ../src/prep_cif.c(226) : error C2059: syntax error : 'type'
    ../src/prep_cif.c(227) : error C2059: syntax error : ')'
    ../src/prep_cif.c(232) : error C2143: syntax error : missing ')' before '*'
    ../src/prep_cif.c(232) : error C2143: syntax error : missing '{' before '*'
    ../src/prep_cif.c(233) : error C2146: syntax error : missing ';' before identifier 'abi'
    ../src/prep_cif.c(234) : error C2059: syntax error : 'type'
    ../src/prep_cif.c(237) : error C2059: syntax error : ')'
    libtool: compile: mv -f "prep_cif.obj" "src/.libs/prep_cif.obj"
    mv: cannot stat 'prep_cif.obj': No such file or directory
    Makefile:1362: ошибка выполнения рецепта для цели <src/prep_cif.lo>
    make[2]: *** [src/prep_cif.lo] Ошибка 1
    make[2]: выход из каталога </cygdrive/d/tmp/libraries/libffi-3.2.1/build-x64>
    Makefile:1596: ошибка выполнения рецепта для цели <all-recursive>
    make[1]: *** [all-recursive] Ошибка 1
    make[1]: выход из каталога </cygdrive/d/tmp/libraries/libffi-3.2.1/build-x64>
    Makefile:730: ошибка выполнения рецепта для цели <all>
    make: *** [all] Ошибка 2
    -----------------------------------------------------------------------------
    what to do - replace symlink by real file.

4) if you forgot to comment FFI_TARGET_HAS_COMPLEX_TYPE
    -----------------------------------------------------------------------------
    ../src/types.c(101) : error C2061: syntax error : identifier '_Complex'
    ../src/types.c(101) : error C2059: syntax error : '}'
    ../src/types.c(101) : error C2065: '_Complex' : undeclared identifier
    ../src/types.c(101) : error C2143: syntax error : missing ')' before 'type'
    ../src/types.c(101) : error C2059: syntax error : ')'
    ../src/types.c(102) : fatal error C1903: unable to recover from previous error(s); stopping compilation
    libtool: compile: mv -f "types.obj" "src/.libs/types.obj"
    mv: cannot stat 'types.obj': No such file or directory
    Makefile:1362: ошибка выполнения рецепта для цели <src/types.lo>
    make[2]: *** [src/types.lo] Ошибка 1
    make[2]: выход из каталога </cygdrive/d/tmp/libraries/libffi-3.2.1/build-x64>
    Makefile:1596: ошибка выполнения рецепта для цели <all-recursive>
    make[1]: *** [all-recursive] Ошибка 1
    make[1]: выход из каталога </cygdrive/d/tmp/libraries/libffi-3.2.1/build-x64>
    Makefile:730: ошибка выполнения рецепта для цели <all>
    make: *** [all] Ошибка 2
    -----------------------------------------------------------------------------

    what to do - comment FFI_TARGET_HAS_COMPLEX_TYPE

5) if you forgot to replace replace two 'jmp short' to 'jmp'
    -----------------------------------------------------------------------------
    ml64 -nologo -c ./win64.asm
     Assembling: ./win64.asm
    ./win64.asm(1029) : error A2075:jump destination too far : by 20 byte(s)
    ./win64.asm(1038) : error A2075:jump destination too far : by 1 byte(s)
    Makefile:1338: ошибка выполнения рецепта для цели <src/x86/win64.lo>
    make[2]: *** [src/x86/win64.lo] Ошибка 1
    make[2]: выход из каталога </cygdrive/d/tmp/libraries/libffi-3.2.1/build-x64>
    Makefile:1596: ошибка выполнения рецепта для цели <all-recursive>
    make[1]: *** [all-recursive] Ошибка 1
    make[1]: выход из каталога </cygdrive/d/tmp/libraries/libffi-3.2.1/build-x64>
    Makefile:730: ошибка выполнения рецепта для цели <all>
    make: *** [all] Ошибка 2
    -----------------------------------------------------------------------------
    what to do - replace two 'jmp short' to 'jmp'
.

tinyxpath building note

Go to link, download & unpack sources.
In tinyxml.h before '#ifdef TIXML_USE_STL' insert '#define TIXML_USE_STL'.
Headers can be copied to ./include directory
Open ./tinyxpath_lib/tinyxpath_lib.vcproj
Choose mode & settings and build.

note of /MT x64 static zlib compiling

Go to http://www.zlib.net/
Download & unpack the latest stable release - it's near phrase "The current release is publicly available here"
Open in text editor 'win32\Makefile.msc', replace -MD to -MT
Open "VS2013 x64 Native Tools Command Prompt", set current directory to unpacked zlib dir.
Exec 'nmake -f win32/Makefile.msc'
After that you will see new files with extentions:

which you can move somewhere.

Friday, June 12, 2015

building libxml2 in visual studio 2013

How to build release & debug x64 static libraries with /MT or /MTd runtime consequently.
Build without dependencies (as I understand here can be somehow optional included icu & zlib libraries).

1) download sources
git clone git://git.gnome.org/libxml2

2) configure and make.
Open visual studio console (if you open x32 console - it builds x32 library, if x64 - it builds x64 library). Go to libxml2/win32 and enter:
cscript.exe configure.js iconv=no compiler=msvc cruntime=/MT debug=no prefix="..\_build_x64_static_release_mt" static=yes
nmake Makefile.msvc libxml install
nmake clean
cscript.exe configure.js iconv=no compiler=msvc cruntime=/MTd debug=yes prefix="..\_build_x64_static_debug_mtd" static=yes
nmake Makefile.msvc libxml install

'debug=yes' means generate unoptimized code and generate debug symbols.
'debug=no' means generate optimized code and not to generate debug symbols.
such 'prefix' values put include/library/binary files into the directory in the root of your libxml2 with good name.

3) If you want to use libxml only as static library - you can define LIBXML_STATIC in it's headers. You need to do it with headers:

threads.h
xmlexports.h
xmlwin32version.h

In these files you need before first occurrence LIBXML_STATIC insert something like
#ifndef LIBXML_STATIC
#define LIBXML_STATIC
#endif

iconv on windows

I've needed to build static x64 library libiconv on windows - found two useful links. Leave it here. On June 2015 this article & project builds last version of iconv library.
Article with describing of building - link.
Project for visual studio, made by this article - link. It already has settings for lot of building variants - and static x64 too.

libiconv is widespreaded unix gnu library for conversion between encodings. iconv - console utility, which uses this library. Mostly I encountered with it in gunwin32 utils - no one util will work without this dll. But also it appears as dependency in some gnu projects - for example, libxml (added: it's appeared that libxml builds successfully without libiconv - don't know why - either icov optional, or iconv embedded into libxml).

Thursday, April 30, 2015

notes of yara 3.4.0 compiling

tools & version:

windows 8.1
visual studio 2013
yara library 3.3.0 or 3.2.0

foreword

Just notes of yara compiling process on windows with visual studio. I will compile without CUCKOO support - bcs I don't need this & don't want to compile yet another library (jansson64.lib)

action

1) unpack archive

2) open solution: .\yara-3.3.0\windows\yara\yara.sln

3) open 'utils.h' -> replace '#define YR_API EXTERNC __declspec(dllexport)' to '#define YR_API EXTERNC' (bcs I don't like exported symbols in my exe files, and link I wanna statically)

4) choose platform & mode

5) set runtime library for all projects (yara & yarac & libyara):
    properties -> c/c++ -> code generation -> runtime library -> /MTd for debug or /MT for release
    (you can select several projects in time - using 'ctrl'+left_mouse_button_click)

6) add to "Preprocessor Definitions" of all projects
    (Properties -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions)
    lines to not conflict with mysql c connector, for example:
strlcpy=libyara_internal_strlcpy
strlcat=libyara_internal_strlcat

7) Go to libyara properties:
    Properties -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions
    and delete CUCKOO from this list.
Then go to libyara properties:
    Properties -> Configuration Properties -> Librarian -> General -> Additional Dependencies
    and delete jansson64.lib from this list.

7) Here are you must choose - you want to compile it with openssl or without.
Why do you need openssl in yara library:
    - Generate an import hash: https://www.mandiant.com/blog/tracking-malware-import-hashing/ (uses define HAVE_LIBCRYPTO)
    - PE module of yara can extract some info from pe digital signature certificate. (uses define HAVE_LIBCRYPTO)
#if defined(HAVE_LIBCRYPTO)
begin_struct_array("signatures");
declare_string("issuer");
declare_string("subject");
declare_integer("version");
declare_string("algorithm");
declare_string("serial");
declare_integer("not_before");
declare_integer("not_after");
declare_function("valid_on", "i", "i", valid_on);
end_struct_array("signatures");
declare_integer("number_of_signatures");
#endif
    - HASH module of yara can calc provide you cryptographic hash functions: md5, sha1, sha256, checksum32 (uses define HASH, appeared in 3.3.0 version)

If you need some of this functionality - you need to build openssl & you need add for all projects:
    - Additional library directory
    - library file of openssl (libeay32.lib on my pc)

If you don't need this functionality
    - delete HAVE_LIBCRYPTO & HASH from "Preprocessor Definitions" of libyara, and insert HAVE_TIMEGM line - else you get undefined type 'tm'.
        Properties -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions
    - delete libeay64.lib from
        Properties -> Configuration Properties -> Librarian -> General -> Additional Dependencies

After that everything will compiles fine.

Possible errors for googling people

warnings:

---------------------------------------------------------------------------------------------
in x86 (32-bits mode) you will get next warnings:
    3>args.obj : warning LNK4075: ignoring '/EDITANDCONTINUE' due to '/OPT:LBR' specification
    2>args.obj : warning LNK4075: ignoring '/EDITANDCONTINUE' due to '/OPT:LBR' specification

---------------------------------------------------------------------------------------------
1>D:\blablabla\yara-3.3.0\libyara\include\yara/object.h(23): warning C4005: 'INFINITY' : macro redefinition
1>          C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\math.h(102) : see previous definition of 'INFINITY'
1>D:\blablabla\yara-3.3.0\libyara\include\yara/object.h(24): warning C4005: 'NAN' : macro redefinition
1>          C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\math.h(106) : see previous definition of 'NAN'

---------------------------------------------------------------------------------------------
1>libeay32.lib(fips_ers.obj) : warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library
1>libeay32.lib(ecp_nistp224.obj) : warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library
1>libeay32.lib(ecp_nistp256.obj) : warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library
1>libeay32.lib(ecp_nistp521.obj) : warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library
1>libeay32.lib(ecp_nistputil.obj) : warning LNK4221: This object file does not define any previously undefined public symbols, so it will not be used by any link operation that consumes this library

---------------------------------------------------------------------------------------------
what to do - ignore them

if you didn't set HAVE_TIMEGM

---------------------------------------------------------------------------------------------
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(23): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(23): error C2227: left of '->tm_year' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(26): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(26): error C2227: left of '->tm_mon' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(27): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(27): error C2227: left of '->tm_year' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(29): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(29): error C2227: left of '->tm_mday' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(31): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(31): error C2227: left of '->tm_hour' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(33): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(33): error C2227: left of '->tm_min' must point to class/struct/union/generic type
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(35): error C2027: use of undefined type 'tm'
1>          d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(14) : see declaration of 'tm'
1>d:\blablabla\yara-3.2.0\libyara\modules\pe_utils.c(35): error C2227: left of '->tm_sec' must point to class/struct/union/generic type

---------------------------------------------------------------------------------------------
what to do - add HAVE_TIMEGM

Wednesday, April 29, 2015

linking problem with C

Story

Recently I've got errors during building project (at linking step), spent a lot of time to solve it - wrote some perl script for solving my problem, but finally found simple and nice solution.
The problem occured when several .lib files stored the same symbol - Visual Studio in this case generates next message:
   1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: socket already defined in ws2_32.lib(WS2_32.dll)
that means:
   - library file Win32_Interop.lib contains symbol 'socket'
   - library file ws2_32.lib contains symbol 'socket'
and linker can't decide which instance must be connected when somewhere called socket().

Examples of such errors (which I met):
    - yara defines symbols (strlcpy,strlcat) which is also defined in mysqlclient.lib
    - redis defines symbols (pthread_create, pthread_cond_init, pthread_cond_destroy, pthread_cond_wait, pthread_cond_signal) which is also defined in mysqlclient.lib
    - redis defines symbols (socket, WSASend, WSARecv, WSACleanup, ioctlsocket, setsockopt, getsockopt, connect, listen, bind, shutdown, htons, htonl, getpeername, getsockname, ntohs, select, ntohl, freeaddrinfo, getaddrinfo, WSASetLastError, WSAGetLastError, WSAIoctl) which is also defined in standard windows library ws2_32.lib

There are simple way - say to linker smthng like "choose them in random way":
'properties -> Linker -> General -> Force File Output' with any of next values:
    "Enabled (/FORCE)"
    "Multiply Defined Symbol Only (/FORCE:MULTIPLE)"
but it can lead to some mystic bugs. So, I don't like that way.

Here are facts which obviously lead to solution:
    - All such disambiguated symbols which I saw - it's only for internal usage in libraries. No one such symbol designed for calling outside.
    - And all such symbols I saw - was defined in one file, and called from another - so, I couldn't just make them 'static'.
    - All these projects I'm building from source code.
And here are obvious solution - if it's internal library stuff - who cares which names these internal symbols have.

So, task became 'replace all instances of lexem in projects to another lexem'.
It's refactoring task - and seems I can use typical refactoring tools. but here are things:
    - paid software (klocwork, Slickedit) - I need to use it only once to create lib file. I don't want to pay for that. And trial sucks.
    - some plugins for some IDE (Eclipse CDT, Xrefactory for Emacs, Netbeans, Qt Creator) - I don't think they have batch mode. I don't want manually click for refactoring. And I don't want install some huge ide.
    - GCC MELT - maybe, but I don't want gcc.
Actually one tool looked nice - cscope - but on the first meeting it looked creepy. And I still didn't find out how it works.

I searched ways to solve it by myself.
Simple s/lexem_A/lexem_B/ obviously won't work because if you want replace WSACleanup in redis to any another lexem - it ruins next constructions:
    auto f_WSACleanup = dllfunctor_stdcall<int>("ws2_32.dll", "WSACleanup");
because string constant will be replaced also.
Also if lexem_A is substring of some another lexem - it also will be affected.

Make lexical analysis is difficult. Especially if you are newbie in this area.

On that time I wrote perl script, which uses some simple heuristic analysis of context. Tested it on 2 projects - it worked fine. But here are simple & nice solution.

Simple and nice solution

Answet is preprocessing.
C/C++ compiler apply defines to lexems - it won't affect on substrings of another lexems; it won't affect on string constants. Ideal solution.
Even more - Visual Studio allows set defines without modifications of any source code. And even more - Visual Studio allows you do it with several projects in time! You can set defines for all projects in solution.
Just select c/c++ projects by clicking on projects with pressed 'ctrl' button (if you select also c# project, for example - nothing will work. Only c/c++ projects must be selected).
Then click right mouse button and go to
"Properties" -> "Configuration Properties" -> "C/C++" -> "Preprocessor" -> "Preprocessor Definitions"
and add what do you want to replace in format lexemA=lexemB. For example - here are my stuff for yara (to not conflict with mysqlclient.lib):
strlcpy=libyara_internal_strlcpy
strlcat=libyara_internal_strlcat

That's it. Apply it & compile - all lexems will be replaced.

Examples

--------------------------------------------------------
redis (don't forget you don't need select c# project - "ReleasePackagingTool"):
pipe=redis_internal_pipe
socket=redis_internal_socket
fdapi_close=redis_internal_fdapi_close
open=redis_internal_open
setsockopt=redis_internal_setsockopt
fcntl=redis_internal_fcntl
poll=redis_internal_poll
getsockopt=redis_internal_getsockopt
connect=redis_internal_connect
read=redis_internal_read
write=redis_internal_write
fsync=redis_internal_fsync
fdapi_fstat64=redis_internal_fdapi_fstat64
listen=redis_internal_listen
ftruncate=redis_internal_ftruncate
bind=redis_internal_bind
shutdown=redis_internal_shutdown
htons=redis_internal_htons
htonl=redis_internal_htonl
getpeername=redis_internal_getpeername
getsockname=redis_internal_getsockname
ntohs=redis_internal_ntohs
ioctlsocket=redis_internal_ioctlsocket
inet_addr=redis_internal_inet_addr
gethostbyname=redis_internal_gethostbyname
inet_ntoa=redis_internal_inet_ntoa
fdapi_fwrite=redis_internal_fdapi_fwrite
fdapi_setmode=redis_internal_fdapi_setmode
WSASetLastError=redis_internal_WSASetLastError
WSAGetLastError=redis_internal_WSAGetLastError
WSAIoctl=redis_internal_WSAIoctl
WSASend=redis_internal_WSASend
WSARecv=redis_internal_WSARecv
WSACleanup=redis_internal_WSACleanup
WSAGetOverlappedResult=redis_internal_WSAGetOverlappedResult
WSADuplicateSocket=redis_internal_WSADuplicateSocket
WSASocket=redis_internal_WSASocket
select=redis_internal_select
ntohl=redis_internal_ntohl
isatty=redis_internal_isatty
access=redis_internal_access
lseek64=redis_internal_lseek64
fdapi_get_osfhandle=redis_internal_fdapi_get_osfhandle
fdapi_open_osfhandle=redis_internal_fdapi_open_osfhandle
freeaddrinfo=redis_internal_freeaddrinfo
getaddrinfo=redis_internal_getaddrinfo
inet_ntop=redis_internal_inet_ntop
accept=redis_internal_accept
pthread_self=redis_internal_pthread_self
pthread_create=redis_internal_pthread_create
pthread_cond_init=redis_internal_pthread_cond_init
pthread_cond_destroy=redis_internal_pthread_cond_destroy
pthread_cond_wait=redis_internal_pthread_cond_wait
pthread_cond_signal=redis_internal_pthread_cond_signal
--------------------------------------------------------
yara:
strlcpy=libyara_internal_strlcpy
strlcat=libyara_internal_strlcat
--------------------------------------------------------

Script

Just put it here: https://github.com/HostageBrain/my_lovely_refactor
It analyzes context and replaces lexems which you needed.
Usage:
   - you are specifying extensions
   - you are specifying lexems for replacing
   - you can optional add custom data types for heuristic (redis required 2 such items)
   - you are running 'my_lovely_refactor.pl path_to_directory'
And it prints files where replacings happened. Original file saved in .N.bak extension, where N - nearest free number.

Errors for googling people

if you are trying compile together redis, mysql & ws2_32
--------------------------------------------------------
1>------ Build started: Project: blablabla, Configuration: Debug x64 ------
1>Win32_Interop.lib(win32fixes.obj) : error LNK2005: pthread_create already defined in mysqlclient.lib(my_winthread.obj)
1>Win32_Interop.lib(win32fixes.obj) : error LNK2005: pthread_cond_init already defined in mysqlclient.lib(my_wincond.obj)
1>Win32_Interop.lib(win32fixes.obj) : error LNK2005: pthread_cond_destroy already defined in mysqlclient.lib(my_wincond.obj)
1>Win32_Interop.lib(win32fixes.obj) : error LNK2005: pthread_cond_wait already defined in mysqlclient.lib(my_wincond.obj)
1>Win32_Interop.lib(win32fixes.obj) : error LNK2005: pthread_cond_signal already defined in mysqlclient.lib(my_wincond.obj)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: socket already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: WSASend already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: WSARecv already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: WSACleanup already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: ioctlsocket already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: setsockopt already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: getsockopt already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: connect already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: listen already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: bind already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: shutdown already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: htons already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: htonl already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: getpeername already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: getsockname already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: ntohs already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: select already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: ntohl already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: freeaddrinfo already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: getaddrinfo already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: WSASetLastError already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: WSAGetLastError already defined in ws2_32.lib(WS2_32.dll)
1>Win32_Interop.lib(Win32_FDAPI.obj) : error LNK2005: WSAIoctl already defined in ws2_32.lib(WS2_32.dll)
1>D:\projects\blablabla\blablabla.exe : fatal error LNK1169: one or more multiply defined symbols found
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

--------------------------------------------------------
what to do - use defines (read note from the beginning)

if you are trying compile together mysql C connector & yara
--------------------------------------------------------
1>------ Build started: Project: blablabla, Configuration: Debug x64 ------
1>libyara64.lib(strutils.obj) : error LNK2005: "unsigned __int64 __cdecl strlcat(char *,char const *,unsigned __int64)" (?strlcat@@YA_KPEADPEBD_K@Z) already defined in mysqlclient.lib(crypt_genhash_impl.obj)
1>D:\projects\blablabla\blablabla.exe : fatal error LNK1169: one or more multiply defined symbols found
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

--------------------------------------------------------
what to do - use defines (read note from the beginning)