315 lines
13 KiB
Org Mode
Executable File
315 lines
13 KiB
Org Mode
Executable File
# -*- mode: org ; coding: utf-8-unix -*-
|
|
# last updated : 2015/07/21.00:46:31
|
|
|
|
|
|
#+TITLE: Clang Server Manual
|
|
#+AUTHOR: yaruopooner
|
|
#+EMAIL: [https://github.com/yaruopooner]
|
|
#+OPTIONS: author:nil timestamp:t |:t \n:t ^:nil
|
|
|
|
|
|
[[./readme.ja.org][Japanese Manual]]
|
|
|
|
* About this document
|
|
This is explained about self-build of clang-server.
|
|
When you use clang-server binary for distribution on Windows environment, it is not necessary to read this document.
|
|
|
|
* Step of self-build
|
|
A build of clang-server requires libclang of LLVM.
|
|
Therefore, first self-build is LLVM libclang, next self-build is clang-server.
|
|
|
|
** LLVM self-build
|
|
You have to do four step of following.
|
|
There is also a script which simplify this work.
|
|
- Checkout LLVM
|
|
- Apply patch to LLVM for libclang
|
|
- Project file generation by CMAKE or configure
|
|
- Build
|
|
|
|
** clang-server self-build
|
|
Patch applied libclang library use by LLVM self-build
|
|
Use the patch applied libclang library
|
|
- Project file generation by CMAKE
|
|
- Build
|
|
- Installation
|
|
|
|
* Software required for self-build
|
|
** Windows
|
|
The following is required.
|
|
*** LLVM
|
|
The following built library is required.
|
|
libclang.lib or libclang.imp
|
|
libclang.dll
|
|
|
|
*** Visual Studio 2015/2013/2012/2010
|
|
any OK
|
|
|
|
*** CMake
|
|
http://www.cmake.org/
|
|
|
|
Download cmake archive, and decompress to any location.
|
|
CMake is used for Visual Studio Solution File generation and build and installation.
|
|
|
|
** Linux
|
|
The following is required.
|
|
*** LLVM
|
|
The following built library is required.
|
|
libclang.so
|
|
|
|
*** CMake
|
|
#+begin_src shell
|
|
$ sudo apt-get install cmake
|
|
#+end_src
|
|
|
|
If you want to use latest version, download from following
|
|
|
|
http://www.cmake.org/
|
|
|
|
e.g.
|
|
Download cmake-3.1.3.tar.gz, and decompress to work directory.
|
|
You perform a build and installation.
|
|
#+begin_src shell
|
|
$ tar -xf cmake-3.1.3.tar.gz .
|
|
$ cd cmake-3.1.3
|
|
$ ./configure && make
|
|
$ make install
|
|
#+end_src
|
|
|
|
* Self-Build
|
|
** Windows
|
|
*** LLVM
|
|
Required process for LLVM self-build.
|
|
And you must apply patch to LLVM before build.
|
|
It isn't necessary install a package after self-build.
|
|
You must designate generated binary PATH after LLVM self-build at CMake project generation argument.
|
|
Therefore clang-server is able to build.
|
|
When LLVM is already installed, you must designate installed directory of LLVM.
|
|
|
|
When you want to self-build,
|
|
Checkout from SVN by yourself, and perform the LLVM Solution File generation and build by cmake.
|
|
Or, use following script.
|
|
https://github.com/yaruopooner/llvm-build-shells
|
|
|
|
**** How to designate the LLVM patch in the script
|
|
You will designate -tasks to the argument of builderShell,
|
|
and designate PATCH to the parameter of -tasks,
|
|
It will gives a table that describes the path to apply the patch and patch file to parameter of -patchInfos.
|
|
Please refer to the sample.ps1 of llvm-build-shells for details.
|
|
|
|
**** The contents of the LLVM patch
|
|
Use of mmap always invalidation.
|
|
|
|
*** clang-server
|
|
Use the ac-clang/build/builder_sample.bat
|
|
Please edit the builder_sample.bat as necessary.
|
|
It's necessary to execute in the command line or Windows Explorer.
|
|
|
|
- example
|
|
#+begin_src bat
|
|
cmake -G "Visual Studio 12 2013 Win64" ../clang-server -DLIBRARY_PATHS="c:/cygwin-x86_64/tmp/llvm-build-shells/ps1/clang-360/build/msvc-64/" -DCMAKE_INSTALL_PREFIX="c:/cygwin-x86_64/usr/local/bin/"
|
|
#+end_src
|
|
|
|
- Option commentary
|
|
- =-DLIBRARY_PATHS=
|
|
You have to designate location of LLVM self-build completed directory.
|
|
It is necessary to designate the directory that a binary was generated.(e.g. {LLVM output path}/Release/)
|
|
If you omit this option, value will be use =ac-clang/clang-server= .
|
|
- =-DCMAKE_INSTALL_PREFIX=
|
|
You have to designate installation location of clang-server.
|
|
If you omit this option, value will be use =C:/Program Files/clang-server= .
|
|
|
|
** Linux
|
|
*** LLVM
|
|
Required process for LLVM self-build.
|
|
And you must apply patch to LLVM before build.
|
|
It isn't necessary install a package after self-build.
|
|
You must designate generated binary PATH after LLVM self-build at CMake project generation argument.
|
|
Therefore clang-server is able to build.
|
|
When LLVM is already installed, you must designate installed directory of LLVM.
|
|
|
|
When you want to self-build,
|
|
Checkout from SVN by yourself, and perform the LLVM Solution File generation and build by cmake.
|
|
Or, use following script.
|
|
https://github.com/yaruopooner/llvm-build-shells
|
|
|
|
**** How to designate the LLVM patch in the script
|
|
You will designate -patch to the argument of executeBuilder.
|
|
Add to -patchApplyLocation the path where you want to apply the patch.
|
|
You write the patch file to -patchPath gives as an parameter.
|
|
-patchApplyLocation,-patchPath becomes the pair, it is possible to multiple times designate.
|
|
Please refer to the sample.sh of llvm-build-shells for details.
|
|
|
|
**** The contents of the LLVM patch
|
|
Use of mmap always invalidation.
|
|
|
|
*** clang-server
|
|
Use the ac-clang/build/builder_sample.sh
|
|
Please edit the builder_sample.sh as necessary.
|
|
Execute the builder_sample.sh
|
|
|
|
- example
|
|
#+begin_src shell-script
|
|
cmake -G "Unix Makefiles" ../clang-server -DLIBRARY_PATHS="/home/yaruopooner/work/llvm-build-shells/sh/clang-350/build" -DCMAKE_INSTALL_PREFIX="~/work/clang-server"
|
|
#+end_src
|
|
|
|
- Option commentary
|
|
- =-DLIBRARY_PATHS=
|
|
You have to designate location of LLVM self-build completed directory.
|
|
It is necessary to designate the directory that a binary was generated.(e.g. {LLVM output path}/Release/)
|
|
If you omit this option, value will be use =ac-clang/clang-server= .
|
|
- =-DCMAKE_INSTALL_PREFIX=
|
|
You have to designate installation location of clang-server.
|
|
If you omit this option, value will be use =/usr/local/bin= .
|
|
|
|
* Patch was applied binary(Windows Only)
|
|
https://github.com/yaruopooner/ac-clang/releases
|
|
|
|
clang-server-X.X.X.zip is you can download from the above
|
|
The archive is 3 files contain, these file applied patch.
|
|
- clang-server.exe
|
|
- libclang.dll
|
|
- libclang.lib or libclang.imp
|
|
|
|
When you want to self-build only clang-server without LLVM,
|
|
clang-server-X.X.X.zip decompress to ac-clang directory.
|
|
Then, it will be placed in the following.
|
|
ac-clang/clang-server/binary/clang-server.exe
|
|
ac-clang/clang-server/library/x86_64/release/libclang.dll
|
|
ac-clang/clang-server/library/x86_64/release/libclang.lib
|
|
|
|
* Restrictions when you use LLVM official libclang without applying a patch
|
|
** A specific file is locked and cannot save it
|
|
When you try to save the edited header file,
|
|
it will be "basic-save-buffer-2: Opening output file: invalid argument `HEADER-FILE-NAME`",
|
|
and you can't save.
|
|
This occur if it meets certain conditions.
|
|
This condition is met when the header file size is larger than 16kB.
|
|
It is not at all occur when header file size is smaller than 16kB.
|
|
This issue belong to TranslationUnit(TU) of libclang.
|
|
The inclusion target file is locked by TU of libclang.
|
|
By performing a provisional transaction in ac-clang side, the more or less is erased, but it can't be avoided perfectly.
|
|
When this issue is occurring, only manual handle can be avoided.
|
|
|
|
*** Solution in Emacs side
|
|
I suppose that combination of source file is foo.cpp/foo.hpp.
|
|
When foo.hpp(modified) can't save, foo.cpp is (modified) often, so foo.cpp have to saved.
|
|
Therefore, foo.hpp should be possible to save.
|
|
When this can't save,
|
|
foo.hpp is included by source files besides foo.cpp, and it has (modified) status.
|
|
You have to save those.
|
|
And, when corresponding source is activated by definition jump feature, even if buffer don't modified that buffer is activated.
|
|
You try remove corresponding buffer, or (ac-clang-deactivate) must be execute in buffer.
|
|
In other cases, when you try save header file that file size larger than 16kB
|
|
When you save a header file of larger than 16kB, if it fails.
|
|
And that header file does not opened.
|
|
In this case, header file is included by a far module from current source file.
|
|
When you having developed a library module framework, it may be easy to occur.
|
|
because library and framework is included from application side.
|
|
|
|
*** Issue(Implementation issues explanation, it wanted suggested solutions)
|
|
When session of "foo.cpp" is edited in the buffer,
|
|
TU continue locking to included header file after parsed "foo.cpp".
|
|
|
|
When you edit and save to "foo.hpp" in this state, it occur error, because file is locked by mmap.
|
|
|
|
Therefore I modified a server as follows.
|
|
So while maintaining the session when "foo.cpp" saving,
|
|
TU is generated when "foo.cpp" is edited after TU released.
|
|
|
|
Therefore "foo.hpp" is possible to save that the included header file is unlocked after "foo.cpp" saved.
|
|
|
|
When a "foo.hpp" is included buffer where exist in buffer editing group without buffer of "foo.cpp",
|
|
the lock is not released when you does not save all them.
|
|
|
|
|
|
In the Windows environment,
|
|
This lock is not open function of I/O, is a lock by CreateFileMapping of WindowsAPI.
|
|
libclang FileManager does allocation to memory mapped file for the files larger than 16kB.
|
|
When TU is released, memory mapped file is released by UnmapViewOfFile, these becomes writable to file.
|
|
|
|
In the Linux environment,
|
|
problems with mmap/munmap bug differ slightly from the Windows environment, but also occurred in Linux environment.
|
|
The method add to "class Foo" in "foo.hpp" in the state that holds TU of "foo.cpp", and save to file.
|
|
After "foo.hpp" update, when you try complete method of "class Foo" in the "foo.cpp", TU will crash.
|
|
in this case, libclang output to STDOUT that "libclang: crash detected in code completion"
|
|
libclang output "libclang: crash detected in code completion" to STDOUT.
|
|
The process of clang-server is living in this situation.
|
|
Completion is possible after deletion of session and creation of session.
|
|
|
|
** Miscellaneous
|
|
The above problems are solved by patching for libclang.
|
|
|
|
When you use the patch applied release binary(libclang.dll or so) it is not occur.
|
|
When you use the patch does not applied to LLVM self-build and LLVM official binary, this problem is occur.
|
|
I think specification bug of clang side. This problem has been reported to LLVM bugzilla. in the corresponding waitting.
|
|
http://llvm.org/bugs/show_bug.cgi?id=20880
|
|
|
|
* Patch commentary
|
|
** Patch
|
|
Use the ac-clang/clang-server/patch/invalidate-mmap.patch
|
|
#+begin_src shell-script
|
|
cd llvm/
|
|
svn patch ac-clang/clang-server/patch/invalidate-mmap.patch
|
|
#+end_src
|
|
|
|
** The contents of the LLVM patch(invalidate-mmap.patch)
|
|
Patch is applied so as not to use mmap.
|
|
Apply to the following source code to =clang-trunk/llvm/lib/Support/MemoryBuffer.cpp=
|
|
|
|
#+begin_src C++
|
|
static error_code getOpenFileImpl(int FD, const char *Filename,
|
|
OwningPtr<MemoryBuffer> &result,
|
|
uint64_t FileSize, uint64_t MapSize,
|
|
int64_t Offset, bool RequiresNullTerminator) {
|
|
#+end_src
|
|
|
|
It is determined availability of mmap for file by shouldUseMmap call from the above function.
|
|
|
|
#+begin_src C++
|
|
static bool shouldUseMmap(int FD,
|
|
size_t FileSize,
|
|
size_t MapSize,
|
|
off_t Offset,
|
|
bool RequiresNullTerminator,
|
|
int PageSize) {
|
|
#+end_src
|
|
When the result of function is always false, mmap is not never used.
|
|
Therefore, the following modify has been applied to the top of this function.
|
|
#+begin_src C++
|
|
#if 1
|
|
return false;
|
|
#else
|
|
/* original codes */
|
|
#endif
|
|
#+end_src
|
|
|
|
** TODO Additional Specification of LLVM3.5
|
|
IsVolatileSize has been added to arguments of shouldUseMmap and getOpenFileImpl.
|
|
This will be passed unchanged to shouldUseMmap.
|
|
|
|
It is executed as follows in the shouldUseMmap top.
|
|
#+begin_src C++
|
|
if (IsVolatileSize)
|
|
return false;
|
|
#+end_src
|
|
|
|
Following comments had been attached
|
|
#+begin_src C++
|
|
// mmap may leave the buffer without null terminator if the file size changed
|
|
// by the time the last page is mapped in, so avoid it if the file size is
|
|
// likely to change.
|
|
#+end_src
|
|
|
|
Although that said, there is a situation which isn't assumed Variously, I'm supposing that mistake of specification.
|
|
Moreover, upstream function of buffer association function
|
|
I was found some place where is not designated value of IsVolatileSize and is used default value of constructor argument.
|
|
I tried modified it.
|
|
Result was become more better than conventional.
|
|
But it seem to have a problem, because I was not able to control mmap like a assumption.
|
|
I'm not enough understand the specification of around the file system and memory of LLVM.
|
|
For that reason that the rightly correction is difficult.
|
|
Therefore, the above patching becomes the most safe way at present.
|
|
|
|
|