@Andy_Huntsman wrote:

TL;DR
What is the recipe or best practice to save a large collection of MutableDocuments to a database?

Back Story
Our code makes an API call to fetch about 11,000 model classes from an API call. We then persist these into a local CBL database, and we are NOT (yet) using a sync gateway. So everything is on the device, and we’re just using CBL as our local database of choice, with the goal of setting up a sync gateway in the future.

The model class has a Couchbase MutableDocument property, which is instantiated by the particular model constructor invoked by the API consumer code, since our model is immutable. I say this so you know that by the time we pass the 11,000 models to our CBL database, the MutableDocument for each model already exists, and the only work to do is to invoke Database.Save(myMutableDocument) for each MutableDocument.

The model class has 23 attributes, most of which are simple value types; only a few are collections which contain fewer than 5 items each.

I’ve done a lot of A/B testing in the code, with and without Database.InBatch, with and without PLINQ operators, but the gist of my code is this:

myMutableDocuments.AsParallel().ForAll(mutableDocument => database.Save(mutableDocument))

Also, we’re doing this in the background, on a brand new, empty database. Only after this temporary database is built do we swap it with the existing database file so that our repository classes can see it.

The 20 seconds is on older devices like iPhone 6, but even on newer devices it’s like 10-15 seconds. That just can’t be right…

Posts: 8

Participants: 4

Read full topic

@victorberger wrote:

Hello,

I am facing problems building Couchbase-lite-C with CMake on Windows. I have also noticed that the LiteCore submodule used in the latest commit of Couchbase-lite-C does not compile on Windows if I try to compile it by itself. The latest commit of LiteCore (master), however, compiles with no problems on Windows.

This is the error message I am currently receiving:

17:07:59: Running steps for project CouchbaseLite_C…

17:08:00: Persisting CMake state…

17:08:17: Starting: “C:\Program Files\CMake\bin\cmake.exe” --build . --target all

Scanning dependencies of target CouchbaseLiteCStatic

[ 0%] Building CXX object CMakeFiles/CouchbaseLiteCStatic.dir/src/CBLBase.cc.obj

CBLBase.cc

[ 0%] Building CXX object CMakeFiles/CouchbaseLiteCStatic.dir/src/CBLBlob.cc.obj

CBLBlob.cc

[ 0%] Building CXX object CMakeFiles/CouchbaseLiteCStatic.dir/src/CBLDatabase.cc.obj

CBLDatabase.cc

[ 0%] Building CXX object CMakeFiles/CouchbaseLiteCStatic.dir/src/CBLDocument.cc.obj

CBLDocument.cc

[ 0%] Building CXX object CMakeFiles/CouchbaseLiteCStatic.dir/src/CBLLog.cc.obj

CBLLog.cc

[ 1%] Building CXX object CMakeFiles/CouchbaseLiteCStatic.dir/src/CBLQuery.cc.obj

CBLQuery.cc

[ 1%] Building CXX object CMakeFiles/CouchbaseLiteCStatic.dir/src/CBLReplicator.cc.obj

CBLReplicator.cc

c:\users\beave\desktop\couchbase-lite-c\src\CBLReplicator_Internal.hh(51): error C3861: ‘max’: identifier not found

c:\users\beave\desktop\couchbase-lite-c\src\CBLReplicator_Internal.hh(55): error C2397: conversion from ‘const uint64_t’ to ‘float’ requires a narrowing conversion

c:\users\beave\desktop\couchbase-lite-c\src\CBLReplicator_Internal.hh(52): warning C4244: ‘initializing’: conversion from ‘const uint64_t’ to ‘float’, possible loss of data

NMAKE : fatal error U1077: ‘C:\PROGRA~2\MICROS~1\2017\BUILDT~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\cl.exe’ : return code ‘0x2’

Stop.

Any help will be appreciated. I had a similar issue about a week ago, which was quickly solved by @jens. Thanks in advance.

Posts: 2

Participants: 2

Read full topic

@Gjek wrote:

Hello

I am trying to do the following with Traefik 1.7.

https://docs.couchbase.com/sync-gateway/2.1/load-balancer.html

The https work. I can’t get the websocke to work.

@simon.murray
2019-11-17T19:12:01.224Z [DBG] WS+: c:[18366323] Push RPY#6
2019-11-17T19:12:01.224Z [DBG] WS+: c:[18366323] Sending frame: RPY#6 (flags= 1, size= 10)
2019-11-17T19:12:01.401Z [DBG] WS+: c:[18366323] receiveLoop stopped
2019-11-17T19:12:01.401Z [DBG] WS+: c:[18366323] parseLoop stopped
2019-11-17T19:12:01.401Z [DBG] WS+: c:[18366323] Sender stopped
2019-11-17T19:12:01.401Z [INF] HTTP: c:[18366323] #36405: --> BLIP+WebSocket connection closed

Posts: 2

Participants: 2

Read full topic

@jda wrote:

I have a data structure where I need to get the result of a query like this (in N1QL):

SELECT * FROM data 
WHERE type='ObservationType' AND "2" in locationtypes

In .Net lite I have this code:

if (!string.IsNullOrEmpty(locationType))
{
       _condition = _condition.And(Expression.String(locationType).In(Expression.Property("locationtypes")));
}

… but I get a: System.NotSupportedException has been thrown. I tried with this as well:

       _condition = _condition.And(Expression.String(locationType).In(Expression.Value(Expression.Property("locationtypes"))));

Same error

How can I make the query understand that I want to use the “locationtypes” property?

This is a sample output from the N1QL query:

      "locationtypes": [
        "2"
      ],

… or:

      "locationtypes": [
        "4",
        "5"
      ],

Posts: 4

Participants: 2

Read full topic

@blimkemann wrote:

How can I reproduce the ORDER BY clause of the following query using QueryBuilder in Coucbase Lite (c#)?

Also, is it possible to do a join with the double pipe (||) in QueryBuilder?

SELECT c.CardName AS CardName, h.CardName AS CardNameH
FROM RedbookCore AS g
LEFT JOIN RedbookCore AS c ON meta©.id = (‘UserCard~’ || g.CardGuid)
LEFT JOIN RedbookEBookH2 AS h ON meta(h).id = (‘Directory~’ || g.CardGuid)
WHERE meta(g).id LIKE ‘UserCardGrouping~%’
ORDER BY UPPER(IFMISSINGORNULL(c.CardName, h.CardName));

Posts: 1

Participants: 1

Read full topic

@Asusralis wrote:

I see posts about LINQ for the server database and a post in '16 using AsQueryable in 1.4, but I can’t seem to find any information about this method. Was LINQ removed from Lite? I’m on 2.6.

Posts: 2

Participants: 2

Read full topic

@Eldhose_Issac wrote:

Hi
I am new to couch database. In the project, we run the couch sync gateway in a windows machine (local only) and have ios mobile app. I require to detect when the ios app goes offline and to set a flag in the database from the windows app. Is there any way by which I can know the mobile device gone offline.
I am not using background sync. And know I got a few seconds to set database changes when the mobile app goes offline. But no way if the app crashes
Thanks in advance
I am using Sync gateway 2.6.1 in windows
And CouchbaseLite-Swift version 2.6.3 (Community Edition)

Posts: 3

Participants: 3

Read full topic

@benjamin_glatzeder wrote:

I’m trying to understand the following logs. Here’s an example:

[INF] CRUD: 	Doc "doc_id / "rev_id(?)" in channels "{}"

Does this mean this document has no channel? My data model always requires a channel for every document. Could this be a delete statement of a document and hence it has no channel?

Here’s a longer excerpt with user id 4b6bae75-1778-4b0b-b56d-5b213b2daae3

2019-11-20T21:57:11.296+01:00 [INF] CRUD: 	Doc "4b6bae75-1778-4b0b-b56d-5b213b2daae3::shoppingListItem::k2qh9t4rq8wt::e5961b0baa36" / "101-5a39e9555ed93fcd82031aea42c0c503b12dabee" in channels "{}"
2019-11-20T21:57:11.300+01:00 [INF] Changes: c:[4378fbc4] 148382919 is the oldest skipped sequence, using stable sequence number of 148382918 for this feed   (to 4b6bae75-1778-4b0b-b56d-5b213b2daae3)
2019-11-20T21:57:11.300+01:00 [INF] Cache: c:[4378fbc4] GetCachedChanges("!", 148336894) --> 0 changes valid from #1
2019-11-20T21:57:11.300+01:00 [INF] Cache: c:[4378fbc4] GetCachedChanges("4b6bae75-1778-4b0b-b56d-5b213b2daae3", 148336894) --> 1 changes valid from #148336895
2019-11-20T21:57:11.333+01:00 [INF] CRUD: 	Doc "4b6bae75-1778-4b0b-b56d-5b213b2daae3::shoppingListItem::k2qh9t4rq8wt::ed5c1c951179" / "35-f76012a9c8a2b9cbc56f831a5d5623feb14a1c2b" in channels "{}"
2019-11-20T21:57:11.333+01:00 [INF] CRUD: 	Doc "4b6bae75-1778-4b0b-b56d-5b213b2daae3::shoppingListItem::k2qh9t4rq8wt::db854236399e" / "51-5f9a76c223ed05719d578f94ae82815c042e40af" in channels "{}"
2019-11-20T21:57:11.334+01:00 [INF] CRUD: 	Doc "4b6bae75-1778-4b0b-b56d-5b213b2daae3::shoppingListItem::k2qh9t4rq8wt::926ad9c36038" / "6-15bae51bae4005b238aeab63153ed92f40137e33" in channels "{}"
2019-11-20T21:57:11.334+01:00 [INF] CRUD: 	Doc "4b6bae75-1778-4b0b-b56d-5b213b2daae3::shoppingListItem::k2qh9t4rq8wt::2ebf306e09d8" / "7-a1a4d37595673c2f9357fcb5b9b19547e7e3a864" in channels "{}"
2019-11-20T21:57:11.334+01:00 [INF] CRUD: 	Doc "4b6bae75-1778-4b0b-b56d-5b213b2daae3::shoppingListItem::k2qh9t4rq8wt::7b8dd9608091" / "22-113a5ec01bccab38fcc886b0cc9e9a668d209ab7" in channels "{}"
2019-11-20T21:57:11.334+01:00 [INF] CRUD: 	Doc "4b6bae75-1778-4b0b-b56d-5b213b2daae3::shoppingListItem::k2qh9t4rq8wt::4ff31dd304f9" / "30-695146e8476bb5a2ea924ce2d5192dea9fc169f4" in channels "{}"
2019-11-20T21:57:11.334+01:00 [INF] Cache:   Deferring #148390249 (1 now waiting for #148390248...#148390248) doc "4b6bae75-1778-4b0b-b56d-5b213b2daae3::shoppingListItem::k2qh9t4rq8wt::db854236399e" / "51-5f9a76c223ed05719d578f94ae82815c042e40af"
2019-11-20T21:57:11.335+01:00 [INF] Changes: c:[4378fbc4] 148382919 is the oldest skipped sequence, using stable sequence number of 148382918 for this feed   (to 4b6bae75-1778-4b0b-b56d-5b213b2daae3)
2019-11-20T21:57:11.335+01:00 [INF] Cache: c:[4378fbc4] GetCachedChanges("!", 148390247) --> 0 changes valid from #1
2019-11-20T21:57:11.335+01:00 [INF] Cache: c:[4378fbc4] GetCachedChanges("4b6bae75-1778-4b0b-b56d-5b213b2daae3", 148390247) --> 3 changes valid from #148390248
2019-11-20T21:57:11.335+01:00 [INF] Cache:   Deferring #148390253 (1 now waiting for #148390251...#148390252) doc "4b6bae75-1778-4b0b-b56d-5b213b2daae3::shoppingListItem::k2qh9t4rq8wt::4ff31dd304f9" / "30-695146e8476bb5a2ea924ce2d5192dea9fc169f4"
2019-11-20T21:57:11.336+01:00 [INF] Changes: c:[4378fbc4] 148382919 is the oldest skipped sequence, using stable sequence number of 148382918 for this feed   (to 4b6bae75-1778-4b0b-b56d-5b213b2daae3)
2019-11-20T21:57:11.336+01:00 [INF] Cache: c:[4378fbc4] GetCachedChanges("!", 148390250) --> 0 changes valid from #1
2019-11-20T21:57:11.336+01:00 [INF] Cache: c:[4378fbc4] GetCachedChanges("4b6bae75-1778-4b0b-b56d-5b213b2daae3", 148390250) --> 2 changes valid from #148390251
2019-11-20T21:57:11.365+01:00 [INF] SyncMsg: c:[4378fbc4] #13: Type:setCheckpoint Client:cp-QcNL5yPdvWqOdI6PdbMv8F8qNLA= Rev:0-200 
2019-11-20T21:57:11.365+01:00 [INF] Sync: c:[4378fbc4] Sent 1 changes to client, from seq 148382918::148390247
2019-11-20T21:57:11.399+01:00 [INF] Sync: c:[4378fbc4] Sent 3 changes to client, from seq 148382918::148390248
2019-11-20T21:57:11.430+01:00 [INF] SyncMsg: c:[4378fbc4] #14: Type:setCheckpoint Client:cp-QcNL5yPdvWqOdI6PdbMv8F8qNLA= Rev:0-201 
2019-11-20T21:57:11.436+01:00 [INF] Changes: c:[4378fbc4] 148382919 is the oldest skipped sequence, using stable sequence number of 148382918 for this feed   (to 4b6bae75-1778-4b0b-b56d-5b213b2daae3)
2019-11-20T21:57:11.436+01:00 [INF] Cache: c:[4378fbc4] GetCachedChanges("!", 148390251) --> 0 changes valid from #1
2019-11-20T21:57:11.436+01:00 [INF] Cache: c:[4378fbc4] GetCachedChanges("4b6bae75-1778-4b0b-b56d-5b213b2daae3", 148390251) --> 2 changes valid from #148390252
2019-11-20T21:57:11.436+01:00 [INF] Changes: c:[4378fbc4] 148382919 is the oldest skipped sequence, using stable sequence number of 148382918 for this feed   (to 4b6bae75-1778-4b0b-b56d-5b213b2daae3)
2019-11-20T21:57:11.436+01:00 [INF] Cache: c:[4378fbc4] GetCachedChanges("!", 148390253) --> 0 changes valid from #1
2019-11-20T21:57:11.463+01:00 [INF] Sync: c:[4378fbc4] Sent 1 changes to client, from seq 148382918::148390251

Here’s an excerpt with a channel:

2019-11-20T22:22:42.016+01:00 [INF] CRUD:   Doc "a48f8ebf-4c62-4616-b362-df995e2eb02c::shoppingListItem::59nzmpq6z2os::e2ecd7b9feef" / "7-cd33830148a970c4729e91111f203a00d75b7121" in channels "{a48f8ebf-4c62-4616-b362-df995e2eb02c}"

I had a user report that under no circumstances specific documents do not synchronize between clients. Other documents do synchronize. So I wonder if it has something to do with the " in channels "{}"

Posts: 1

Participants: 1

Read full topic

@giles wrote:

Hi,

Our SSL certificates need to be updated every three months. Is it possible (via the REST API) to tell Sync Gateway to reload the certificates without restarting the service? I would like to avoid SG downtime if possible.

Thanks,
Giles

Posts: 3

Participants: 2

Read full topic

@jda wrote:

I have received feedback from Apple where they reject my app for testing. The reason I put it here is that it seems to have to do with sockets/websockets - that I know Couchbase use for sync.'ing with the sync.gateway…

Perhaps this message can help someone here clarify if Couchbase Lite is causing the issue:

Dear Developer,
We identified one or more issues with a recent delivery for your app, “Fangstjournalen v.2” 2.0.0 (6). Please correct the following issues, then upload again.
ITMS-90338: Non-public API usage - The app references non-public selectors in Angler.iOS: applicationWillTerminate, ddSetLogLevel:, localTarget, newSocketQueueForConnectionFromAddress:onSocket:, setOrientation:animated:, socket:didConnectToHost:port:, socket:didReadPartialDataOfLength:tag:, socket:didReceiveTrust:completionHandler:, socket:didWritePartialDataOfLength:tag:, socket:shouldTimeoutReadWithTag:elapsed:bytesDone:, socket:shouldTimeoutWriteWithTag:elapsed:bytesDone:, socketDidCloseReadStream:, socketDidSecure:, terminateWithSuccess, webSocket:didReceiveMessage:, webSocketDidOpen:. If method names in your source code match the private Apple APIs listed above, altering your method names will help prevent this app from being flagged in future submissions. In addition, note that one or more of the above APIs may be located in a static library that was included with your app. If so, they must be removed. For further information, visit the Technical Support Information at http://developer.apple.com/support/technical/
Though you are not required to fix the following issues, we wanted to make you aware of them:
ITMS-90809: Deprecated API Usage - Apple will stop accepting submissions of apps that use UIWebView APIs . See https://developer.apple.com/documentation/uikit/uiwebview for more information.
Best regards,
The App Store Team

I know the last “error” is not related. Just wanted to bring the entire message.

Thanks for any input in advance!

Posts: 8

Participants: 3

Read full topic

@theunsa wrote:

Hi

My Couchbase (CB) model represents multiple companies, each with multiple users. The companies typically have 10k-100k documents each. All docs are in one bucket.

I have both a web app and a mobile app. Web app talks through Couchbase Sync Gateway (CSG) using the RESTful API and mobile makes use of Couchbase Lite, so also talks through CSG. Only changes that might bypass CSG is those made using CB’s admin console but they are few.

Given the large amount of documents in CB, I do not want it all “tracked” in CSG. Setting “enable_shared_bucket_access=true” and “import_docs=true” replicates all CB docs in CSG which is unnecessary. A lot of the documents will not change that often, if ever so it feels like a waste to keep them from bogging the CSG’s hardware resources.

Now I’ve read a little about “import filters” so I guess there might be a way to filter imports on some datetime field or so but I will then still have the use case of the new mobile user who expects all his/her company data to be on the mobile device when he/she signs up. Meaning my mobile app would then not be able to sync from only the CSG but I will also have to then manually pull from CB the docs not covered by the configured CSG’s datetime import filter.

Currently using:

• CB Server = v6
• CSG = v2.6.1

With that background, a couple of questions:

1. Various forum examples mention “import_docs=continuous” and it seems to work, yet the current documentation (https://docs.couchbase.com/sync-gateway/2.6/config-properties.html) indicates it is a boolean with only true/false. Where can I find the documentation explaining the “continuous” import_docs setting?
2. What is the recommended approach for handling sync with CSG when your CB server has got millions of documents, some of which might not change often?

Posts: 1

Participants: 1

Read full topic

@DougWhitehead wrote:

Its hard to get started. I am using “implementation ‘com.couchbase.lite:couchbase-lite-android:2.6.0’”
I am supposed to evaluate this tech, but I can’t seem to get to square one…
Any suggestion is appreciated.

From what I read from the docs the following should work. But the new Database( dbname, config) fails
dbname is “salesfloor”

And while I am new to Android development, as an old Java guy, I would have thought the exception would have been caught (but alas no, the app just exits).

public void initCouchbaseLite(Context context) {
    CouchbaseLite.init(context);
    appContext = context;
}

// -------------------------
// Database operation
// -------------------------

private void openDatabase(String dbname) {
    DatabaseConfiguration config = null;
    try {
        initCouchbaseLite(getApplicationContext());
        config = new DatabaseConfiguration();
    } catch ( Exception e1 ) {
        setStatus( e1.getMessage() );
    }

// config.setDirectory(String.format("%s/%s", appContext.getFilesDir(), dbname));
try {
database = new Database(dbname, config);
} catch (Exception e) {
setStatus(e.getMessage());
}
int i = 0;
}

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
setStatus(“Not Connected”);
findViewById(R.id.sync).setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
startSession(DATABASE_NAME, DATABASE_NAME);
}
});
}

Posts: 4

Participants: 3

Read full topic

@DougWhitehead wrote:

I have a line of thinking that I’d like to put out there to solicit comment or feedback on things I may have overlooked/misunderstood.

I have a CB server with a Gateway and a bucket with some 700K docs in it. Presumably there will be a few such buckets. These buckets will need to get replicated to handheld devices.

I just managed to get my first sync running and it took over 30 mins to go from 0 to 700K docs. (I haven’t found a bulk replication API yet…) Further, there will be multiple applications in the handheld that will share the same CBL (so that we don’t have multiple copies of the same bucket on the same device).

Does it make sense to run a webserver on the handheld as a means of servicing requests from localhost apps? If so, I would imagine I could implement the methods by calling the Rest APIs on the sync gateway while the data is loading to the local CBL.

In other words for the first hour (or so) the handheld apps would be fully functional when connected to WiFi. And after the sync is complete, the handheld can revert to normal responses from the local CBL.

All of the handhelds at the same store location would sync to the same buckets/channels so after the initial sync things should be quick (assuming you didn’t switch stores which would require a delete and initial sync).

Does this make sense? Is this something others have tried?

Thanks,
Doug

Posts: 1

Participants: 1

Read full topic

@Eldhose_Issac wrote:

This question is out of curiosity. I know walrus mode is only for testing and development,
I configured testing teams sync server in walrus mode and without any couch server now would like to do a search in the sync server system, I have access to sever but no way to search using a mobile device. So is there any way to search in sync-gateway in walrus mode?

Posts: 1

Participants: 1

Read full topic

@DougWhitehead wrote:

If I have a CBL db already synced on the device, how do I delete that so that I can start the sync over? One might need to get different data (new user? new locale? etc.).

I tried stopping replication and deleting the database. But have not figured out how to remove the local storage so it can start over fresh.

Posts: 4

Participants: 2

Read full topic

@jasonwhetton wrote:

Hi,

I get an exception when trying to add an index to an existing couchbase lite database - with a File IO error.

Is this normal? Is it only possible to create indexes on new databases?

Thanks!

Posts: 2

Participants: 2

Read full topic

@paolo.morgano wrote:

I’m trying to evaluate performance of an inner join between documents of a CB Lite 2.x db. Which is the better choice for joining documents?

1. DocA has a property with DocB id
2. DocA has a property with the value of an equivalent property of DocB
3. DocA has a property with the value of an equivalent property of DocB and that property of DocB is indexed

Any help appreciated.
Paolo

Posts: 1

Participants: 1

Read full topic

@itssrinadh wrote:

Hi There,
I have a following a document list and I have a following N1QL query and I need to build an equivalent Query Builder in iOS but I don’t know how to do with ORDER By with multiple conditions.

N1QL Query:

SELECT * from fruits WHERE type = ‘fruits’ ORDER By SUBSTR(createdDate,0,10) DESC, percentChange DESC
LIMIT 10 OFFSET 0

[

{
"name" : "Apple",
"percentChange" : 20,
"type" : "fruits"
"createdDate": "2019-12-04T05:44:12.127Z",

},
{
"name" : "Orange",
"percentChange" : -20,
"type" : "fruits"
"createdDate": "2019-12-04T04:44:12.127Z"

},
{
"name" : "Apricot",
"percentChange" : 10,
"type" : "fruits"
"createdDate": "2019-12-02T04:44:12.127Z"

},
{
"name" : "Avocado",
"percentChange" : -0.1,
"type" : "fruits"
"createdDate": "2019-12-04T04:44:12.127Z"

}
]

Posts: 3

Participants: 2

Read full topic

@Jonathan_G wrote:

I have 2 sync gateways in 2 different data centres - SG1 and SG2. Ideally I want an active \ active sync setup where any client can connect to either SG at any time.

If I use SG-Replicate to keep these 2 in sync, the sequence numbers differ from SG1 to SG2
(ie: on SG1, seq 100 = record A.
on SG2, seq 100 = record B)

Here is the use-case

1. Client1 connects to SG1 @ last_sequence = 100
2. SG1 becomes unavailable so Client1 connects to SG2 @ last_sequence = 100.
3. The sequence numbers on the 2 SG’s are not in sync with each other so Client1 does not get the right records back.

I can not find documentation explaining how to use SG-Replicate in order to have 2 sync gateways which replicate with each other and can be accessed by any of the clients interchangeably. Can someone please explain this to me?

Posts: 1

Participants: 1

Read full topic

@glenn wrote:

Hi,

Is there a way to learn whether a document is rejected by sync gateway in cb lite side?

Let’s say we have a sync gw function which validates the json fields and the data that pushed from client is not valid. As a result, it is simply rejected. In this case, we may want to delete the invalid version of the document from cb lite db for that specific client. Is it possible to do that?

Thanks.

Posts: 2

Participants: 2

Read full topic